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ABSTRACT 


Some  methods  of  analyzing  experimental  data  for 
complex  targets  and  reverberation  are  outlined, 
including  a  heuristic  discussion  of  what  con¬ 
stitutes  an  ensemble  and  how  to  obtain  one 
experimentally.  The  problem  of  then  determining 
whether  or  not  the  actual  data  form  a  valid  ensemble 
is  considered.  Tests  for  the  stability,  or  station¬ 
er  ity,  of  the  underlying  random  mechanism  are  briefly 
described  (test  of  independence  and  homogeneity); 
testa  of  whether  or  not  a  particular  data  set  belongs 
to  some  postulated  probability  distribution  are  pro¬ 
vided  (goodness-of-fit),  including  a  powerful  test 
for  establishing  the  normality  or  non-normality  o^ 
the  sample.  Among  the  tests  considered  are  the  x‘~} 
the  Kblmogorov-Srairnov,  the  runs  test,  and  the  W-test 
for  normality.  These  tests  are  carried  out  for  some 
hypothetical  reverberation  data  tr  illustrate  the 
individual  testa,  at  particular  ra»iges,  which  can 
include  the  domain  of  a  target 'in  the  reverberation. 
Some  second-order  properties  of  various  classes  of 
second  moments  of  these  data  are  also  discussed,  and 
an  approach  to  relating  simulated  data  to  those  from  a 
real  environment  is  briefly  sketched.  This  memorandum 
is  intended  as  a  preliminary  guide  to  the  statlstlced 
treatment  of  data  that  are  obtained  in  target  and 
background  measurements.  As  a  subsequent  task,  auid- 
Itlonal  tests  suid  techniques  remain  to  be  chosen  for 
this  class  of  problems,  including  the  explicit  analysis 
of  data  already  obtained. 
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I.  Introduction*: 

The  purpose  of  this  memorandum,  the  second  in  a  continuing  series, 
is  to  provide  a  number  of  statisticsd  measures  of  experimental  data,  here 
obtained  in  the  study  of  the  simulated,  modeled,  and  actual  versions  of 
complex  targets,  observed  in  a  reverberatory  or  and) lent  noise  environment. 

The  same  types  of  statistical  measures  eire  to  be  similarly  applied  to  the 
background  noise  and  reverberation  themselves.  Since  target  data  and 
reverberation  are  nonstationary  phenomena,  as  the  background  (ambient) 
noise  may  be  on  occasion,  meaningful  results  must  be  obtained  from  a 
suitably  defined  set  of  observations:  measurements  on  a  single  member 
(representation)  of  the  set  are  usually  quite  inadequate.  Furthermore, 
for  obvious  practical  reasons  of  stability  Md  economy  it  is  not  possible 
to  generate  the  theoretically  desirable  infinite  set:  we  are  necessaurlly 
limited  to  the  generation  of  seta  of  finite  size  and  representations  of 
finite  duration. 

Before  we  can  proceed  in  a  technically  clear  fashion,  we  must  define 
a  number  of  fundamental  terms,  whose  meanings  are  not  necessarily  identical 
with  those  of  more  conventional  statistical  usage,  but  which  are  quiet 
common  to  statistical  communication  theory  (Ref.  l) .  Here  we  shall  refer 
to  the  set  of  data,  or  observations, sis  an  ensemble .  euid,  furthermore,  we  shall 
say  that  an 

ensemble  =  a  set  of  objects,  data,  observations,  etc.,  which  possess 
statistical  regularity,  i.e. .  to  which  some  probability 
mesisure  csm  be  mesuiingfully  assigned.  The  probability 
measure  in  question  is  usually  a  probability  distribution 
of  one  (or  more)  of  the  defining  attributes  of  the  set, 
for  example,  magnitude. 


*Portions  of  the  material  in  this  memorsuidum  are  based  on  some  results  of 
the  author's  current  work  under  Contract  Nonr-U885(00)  with  the  Office 
of  Naval  Research. 
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Our  term  "ensemble"  here,  accordingly  implies,  reproduceable,  or  invariant 
statistical  properties .  Similarly,  a  random  process  is  an  ensemble  of 
functions  (or  representations)  of  time  (and  often,  space,  as  well)  where, 
of  course,  the  underlying  /.lechanism  producing  these  functions  is  random 
in  some  sense.  A  theoretical  ensemble,  when  descriptive  of  a  random 
process  like  reverberation  here,  consists  of  an  infinite  number  of  re¬ 
presentations,  or  elements.  An  experimentally  obtained  ensemble,  of  course, 
can  contain  only  a  finite  number  of  representations.  In  conventional 
statistical  useige  the  theoretical  ensemble  is  equivalent  to  the  (infinite) 
"population,"  while  the  finite,  or  "experimental"  ensemble  is  usually 
called  the  "sample."  In  what  follows,  the  nature  of  this  terminology  will 
be  quite  clear.  For  further  details,  see  Ref.  1. 

Key  questions  accordingly  are: 

A.  Do  we  have  a  valid  enseinble,  or  sample,  to  begin  with;  i.e. . 
does  the  set  of  representations  we  have  constructed,  or  observed,  truly 
represent  an  ensemble,  and  hence  can  we  legitimately  deduce  from  it  the 
various  desired  statistical  properties  of  these  data?  In  technical 
Danguage,  is  the  set  (or  sample)  "homogeneous"  and  its  members  " independently 
generated?" 

B.  Is  the  enseinble  stationary  or  nonstationary? 

C.  What  probability  distribution  best  "fits,"  or  "describes" 
within  preset  bounds,  the  statistical  process  in  question? 

Other  questions  are: 

D.  What  is  the  effect  of  sample  duration,  (the  duration  of  each 
member  or  representation  in  the  sample)  and  ensemble  size  (the  number  of 
representations , or  members,  in  the  sample)  on  the  accuracy  with  which  various 
statistics,  e .g. .  means.  Intensity,  covariances,  etc.,  can  be  measured? 
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E.  How  closely  (in  time  or  space)  can  one  sample  the  data  (of 
each  representation)  and  still  retain  statistical  independence  of  the 
sampled  values?,  and  other  questions. 

To  answer  these  we  turn  to  the  appropriate  statistical  methods,  and 
in  particular  to  those  that  put  the  least  premium  on  prior  knowledge.  Many 
sources  for  this  are  available,  and  are  described,  for  example,  in  the  books 
by  Cramer  (Ref.  2),  Hoel  (Ref.  3),  Mood  (Ref.  U) ,  Wilks  (Ref.  5),  Fraser 
(Ref.  6),  and  others  (Ref.  7).  A  particularly  useful  text,  describing  the 
applied  properties  of  some  of  the  more  important  statistical  techniques, 
is  that  of  Siegel  (Ref.  8),  from  which  we  shall  take  some  of  the  needed 
tables  (Appendix,  end  of  this  memorandum)  .  Our  general  task  here  is  thus 
twofold:  (1),  to  select  appropriate  statistical  measures  and  tests  for 
answering  questions  like  the  above  about  our  data;  and  (2),  to  show  in 
detail  how  these  measures  and  tests  eure  to  be  specifically  applied. 

Let  us  state  the  basic  questions  more  fully,  and  arrange  them  in  a 
hierarchy  of  descending  order  of  generality  (although  not  necessarily  of 
practical  importance) ,  by  means  of  which  we  attempt  to  interpret  our  data 
statistically.  Our  ordering  becomes: 

A.  Tests  for  independence  of  the  sampled  data:  essentially,  the 
question  here  is  "are  the  individual  members  of  the  "ensemble"  independent, 
so  that  the  sampled  data,  from  member  to  member,  can  be  treated  as  represent¬ 
ative  of  independent  random  variables,  with  the  same  distribution?"  (See 
pp.  195-198>  Wilks  (Ref.  5).)  This  is  really  the  joint  question  of  (l) 
"homogeneity"  of  the  ensemble— i.e. ,  of  whether  or  not  the  members  are 
generated,  by  the  same  underlying  mechanism,  and  (2). that  the  members  are 
independently  generated. 

Thus,  we  have  the  equivalent; 

Tests  for  the  existence  of  the  ensemble;  fundamentally  we  must 
establish  whether  or  not  our  set  of  observations  (of  size  M,  j  =  1,...,  M) 
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constitutes  a  valid  ensemble.  Tests  of  this  property  are  knovm  as  tests  of 
homot^ene ity  (of  the  ensemble),  and  independence  (of  the  sample  members)  where 
we  wish  to  show  that  our  data  can  be  considered  as  generated  independently 
by  the  same  statistical  mechanism.  Thus,  one  has  here  a  test  of  the 
"stability"  of  the  underlying  mechanisms  as  the  members  of  the  ensemble 
are  interchanged, for  all  times  in  the  interval  (t^^,  t^+T)  during  which  each 
member  is  defined;  (see  Sec.  50-6,  Ref.  2,  for  example). 

B.  Tests  for  "goodness-of-fit;"  in  this  case  we  desire  to  determine 
whether  or  not  our  set  (of  M  random  members)  belongs  to  a  specific  dis¬ 
tribution,  e.g. ,  gauss  (for  high-density  reverberation),  Poisson,  or  others 
(such  as  those  theoretically  predicted  for  envelope  and  phase  in  the  case 
of  narrowband  illumination  (Ref.  9)>  for  example).  VEurious  tests  for 

this  are  discussed  in  Ref.  2,  Sec.  (30.1)— Secs.  (30.^+) ,  (30.8) ;  see  also 
the  nonparametrlc  tests  of  Chap.  lU,  Ref.  5,  esp.  lU.2(b);  Sec.  (1U.3), 
order  statistics,  etc.),  and  p.  k60,  and,  in  particular,  Siegel  (Ref.  8). 
Some  others  are  considered  here  below.  We  can  also  test  for  the  a  priori 
probability  of  an  event's  occurrence:  e.g. ,  signal  in  noise  vs.  noise  alone, 
with  tests  of  this  type. 

C.  First-  and  Second-Order  Sample  Statistics:  here  we  wish  to 
determine  the  dependence  of  the  mean  auid  variance  of  the  sample  mean  and 
variance,  covariance,  etc.,  on  sample  duration  and  ensemble  (l.e. ,  "sample") 
size,  and  on  the  (infinite  population)  pareuneters  of  the  corresponding 
theoretical  ensemble.  In  addition,  we  wish  to  examine  the  dependence 

and  independence  of  samples  on  sampling  intervals.  (See  Secs.  8.1,  8.2, 
of  Wilks  (Ref.  5).) 

We  distinguish  vaurious  types  of  tests,  accordingly  as  the  samples 
are  large  or  small .  or  the  teats  themselves  are  distribution-free,  or 
"nonparametrlc"  (Refs.  6  and  8).  In  these  latter,  we  avoid  the  often 
unsupportable  assumption  of  a  particular  or  specific  dlsti Ibution,  with,  of 
course,  an  attendant  loss  of  efficiency  in  deciding  an  hypothesis  for  a 
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given  sample-size  vis-a-vis  the  case  of  known  distributions.  Tests  of 
homogeneity  (cf.,  A,  above)  may  also  be  applied  to  situations  involving  a 
known  form  of  the  distribution,  but  unknown  distribution  parameters — usually 
the  mean  and/or  variance.  Thus,  one  can  ask  for  tests  of  homogeneity  of 
the  means,  or  the  variances,  etc.  Other  pertinent  questions  in  the  analysis 
and  interpretation  of  data  are:  (l),  confidence  limits  and  regions; 

(2),  the  distributions  of  the  sample  statistics  (e.g. ,  means,  variances, 
etc.,  cf. Wilks  (Ref.  5),  Sec.  (8.3)  et  seq.);  (3),  stationarity  and  non- 
stationarity:  if  the  process  appears  to  be  nonstationary,  is  this  non- 
stationarity  removable  by  a  model  based  on  the  concept  of  stationary  in¬ 
crements?  (See  Yaglom  (Ref.  10),  Sec.  (18);  also  Sec.  (l^).) 

In  the  present  treatment  we  are  not  concerned  with  the  derivation  of 
statistical  tests,  nor  with  such  important  questions  as  what  is  a  "best" 
test.  We  intend  here  merely  to  summarize  and  illustrate  a  few  useful 
results,  and  quote  any  pertinent  properties  they  may  possess,  referring  the 
reader  to  the  statistical  literature  for  the  analytical  details  (Refs.  2, 

5,  and  6).  Nor  is  any  special  originality  claimed  in  this  respect  here, 
with  the  possible  excention  of  actual  application  to  our  present  class 
of  problems.  Other  tests  and  measures  will  be  introduced  as  the  work 
progresses.  What  we  shall  do,  however,  is  to  provide  enough  "mechemism" 
for  the  reader  to  make  his  own  applications  to  specific  problems. 

We  emphasize  that  in  the  kinds  of  problems  we  are  dealing  with  here— 
the  measurement  of  random  phenomena— targets  and  background  noise--it  is 
essential  to  establish  the  reliability  of  these  data:  pertinent  results  are 
ensemble  measures,  not  individual  measures,  so  that  it  is  vital  to  determine, 
within  acceptable  bounds,  that  we  have  a  valid  ensemble  and  that  useful 
meeisures  can  be  obtained  from  it.  The  fact  that  we  have  to  deal  with  finite 
and  even  small  samples  makes  this  a  definite  problem. 
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Accordingly,  we  shall  begin  in  Sec.  II  with  a  brief  discussion  of 
statistical  tests,  both  conditional  and  unconditional;  Sec.  Ill  is  devoted 
to  the  question  of  the  existence  of  an  ensemble,  given  certain  data;  Sec. 

TV  is  concerned  with  a  number  of  procedures  for  "goodness-of-flt:"  do  these 
data  belong  to  a  specified  distribution?  Section  V  discusses  various 
properties  of  the  sample  statistics.  At  present  we  consider  both  small- 
and  large-sample  statistics.  Section  VI  suggests  how  statistical  measures 
may  be  used  to  connect  simuletlon  with  reality,  and  in  Sec.  VII  we  conclude 
with  a  short  summary  of  the  principal  results  and  their  interpretation. 

II .  Some  Brief  Remarks  on  Statistical  Tests; 

The  problems  outlined  above  are  all  examples  of  the  statistical  test 
of  one  hypothesis  against  another.  Let  us  review  very  briefly  the  salient 
features  of  such  binary  ("two-alternative")  tests.  Generally,  we  have 

r 

the  "null"  or  "true"  hypothesis  vs  the  alternative  to 

states  that  the  data  sample  in  question  belongs  to  the  assumed 
population;  or  "true"  hypothesis  state. 

(2.1) 

the  alternative  hypothesis  to  which  states  that  is  not 
"true,"  viz. that  the  data  sample  belongs  to  some  other  population,  or, 

equivalently,  represents  some  other  alternative  hypothesis. 

V. 

On  the  basis  of  a  data  sample,  X  =  (X,,...,X  ),  where  the  X.,  j  =  1,...,  n, 
are  n  observations  (e .g. ,  measurements)  of  a  quantity  X,  we  are  asked  to 
decide  whether  this  data  sample  belongs  to  H^,  or  alternatively,  to  — or, 

Ir  other  words,  on  the  beisls  of  the  observation  X,  whether  or  not  H  is 
"true."  For  this  purpose  a  test  statistic,  Z  =  constructed  and 

compared  with  a  prechosen  threshold,  Z^.  If  ^^ggmple)  ^  reject  H^, 

and  accordingly  accept  the  alternative  and  conversely,  if  ^  Z^, 

we  say  that  is  "true,"  rejecting 
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Of  course,  neither  of  these  decisions  is  always  correct,  since 

X  (and...Z(X))  are  random  variables.  Two  types  of  error  can  occur  in  any 

one  decision  "decide  H  or  "decide  H,  These  are  known  as  Type  I  and 

o  j. 

II  errors,  where  is  rejected  (and  accepted)  when  is  the  true 

state,  and  where  is  decided  when  is  the  true  state,  respectively. 

Let  us  now  determine  these  error  probabilities  for  the  statistical  test 

of  H  vs.H,  described  above.  For  this  we  need 
o  1 

Wi(z|Ho)  =  p.d.  of  the  test  statistic  '^(X),  under  the  hypothesis 
Wi(Z|Hi)  =  p.d.  of  the  test  statistic  Z(X) ,  under  the  hypothesis 


(2.2) 


Our  teat  of  H  against  the  alternative  H,  is  for  any  Z  =  Z(X) , 
o  1  M 


decide  if  Z  S  Z^;  or,  decide  if  Z  >  Z^. 


(2.3) 


The  situation  is  sketched  in  Fig.  (2.1)  for  the  ensemble  of  possible  test 
statistics,  Z,  corresponding  tc  the  ensemble  of  possible  data  vectors  X. 

If  Z  >  Z^  we  decide  but  there  will  be  a  Type  I  error  probability  a 
that  this  decision  is  incorrect,  and  that  these  data  are  really  representative 
of  the  H  state.  Similarly,  if  Z  S  we  decide  H  ,  but  there  will  be  a 
Type  II  error  probability  3  that  this  decision  is  also  incorrect,  and  that 
these  data  really  belong  to  the  state  This  situation  is  shown  in 
Fig.  (2.1).  With  the  help  of  Eq.  (2.2)  we  can  accordingly  say  that 


-j  Wi(Z|H„)lZ  =  f„(Z>V 


(2.1.) 
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where  P^,  are  probabilities  under  respectively.  The  quantity 

q:  is  called  the  significance  level  of  the  test  vs.  the  interval 
Z  5  is  the  acceptance  region,  while  Z  >  Z^  is  the  rejection  or  critical 
region.  The  quantity 


m 

3  W^(z|H^)dZ  =  Pi(Z>Z^) 


(2.6) 


“a 


is  termed  the  power  of  the  test.  Thus,  in  the  test  (2.3)  we  say  that  if 
^sample  ^  ^o  fS’lse  (and  Is  true)  at  significance  level  a,  l.e. , 

the  test  ^  significant  (H^  false)  at  level  a.  On  the  other 

hand,  if  Z. . S  Z..,  we  say  the  test  of  H  (vs.H, )  is  consistent,  at  level 

SdJUpxQ  Q*  OX 

with  the  hypothesis  H^.  Ideally,  we  would  like  to  choose  a  test 
statistic,  Z,  -to  make  both  a  and  p  indefinitely  small,  but  this  is  not 
possible  for  finite  sample  sizes  (n<»).  A  good  test  of  H^(vs.H^)  is  one 
in  which  a,  say,  is  fixed,  and  the  power  of  the  test,  cf,  Eq.  (2.6)  is  large, 
or  equivalently,  the  Type  II  error  probability  3  is  small.  A  "best"  test 
here  is  one  where  3  is  made  as  small  as  possible,  i.e. ,  the  power  of  the 
test  is  maximized  (for  given  sample  size),  with  the  Type  I  error  probability 
fixed.  In  general,  increasing  the  sample-size  increases  the  power,  1-3,  of 
the  test. 


In  actual  practice  our  data  X  have  a  certain  a  priori  probability,  q, 
of  belonging  to  the  null  or  hypothesis  class,  and  conversely,  an  a  priori 
probability,  p  =  1-q,  of  belonging  to  the  state  Hj^.  (if  q  =  1  we  would  know 
for  sure,  a  priori,  that  X  belonged  to  the  process  described  under  and 
no  test  of  vs.  H^,  would  be  necessary.  Similarly,  for  p  =  1,  X  is  known 
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surely  a  priori  to  belong  to  and  no  test  would  again  be  needed.)  The 
error  probabilities  a,  0,  and  the  power  of  the  test,  1  -  P,  are  all 
conditional  probabilities—conditional  on  the  "true"  state  of  affairs 
(i.e.,  on  or  . 

Thus,  we  have  more  explicitly 

a  =  O^H^IHq)  =  (conditional)  Type  I  error  probability  of  deciding  ' 
when  is  really  true, 

P  =  P(H^|h^)  =  (conditional)  Type  II  error  probability  of  deciding 

H  ,  when  H,  is  true, 
o'  1 

(2.7) 


Iqa  =  unconditional  probability  of  deciding  when  is  true;  I 

pP  =  unconditional  probability  of  deciding  when  is  true. 

(2.7a) 

It  cem  be  shown  (Ref.  11)  for  the  "best"  test  in  the  Neyman-Pearson 
sense  of  fixing  a  euid  minimizing  P,  i.e. , 


min(pP  +  \qa)  s  Rjp 


(2.8) 
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(where  6  is  the  rule,  analogous  to  Eq,.  (2.3),  for  making  decisions,  and 
X  is  a  Lagrange  multiplier),  that  the  optimum  test  statistic  Z*  here  is  the 
likelihood  ratio 


and  the  test  itself  is 

decide  H^,  if  Z*  S  Z^(=  ^) ;  or,  decide  H^,  if  Z*  >  Z^(=  ^) ;  H  =  p/q. 

(2.10) 

The  threshold  Z^  is  now  seen  to  be  a  ratio  of  X  and  the  a  priori  probability- 
ratio  p;  X  Itself,  in  the  more  general  sense  of  statistical  decision  theory 
(Ref.  1,  Chap.  l8),  is  a  cost  ratio.  Thus,  our  threshold  Z^  embodies  both 
the  subjective  elements  of  cost  assignment  (value  judgements)  auid  priori 
probabilities,  and  takes  account  of  the  fact  that  the  occurrence  of  the 
particular  data  X  at  hand  is  a  priori  weighted  vis-a-vis  and  Similar 
remarks  apply  for  sub-optimiom  tests,  where  Z  Z*  now:  the  threshold  Z^ 
still  embodies  X  and  p. 

Finally,  the  validity  of  a  test,  i.e. .  that  it  yield  meaningful 
results  on  the  basis  of  the  experimental  data  euid  ohe  significance  level 
chosen--depends  on  randomization  and  replication  (Ref.  12).  By  random¬ 
ization  is  meant  effectively  the  generation  of  a  true  ensemble  in  the  process 
of  data  production,  cf.  the  remarks  A.,  Sec.  1,  above.  Replication  is  the 
generation  of  sufficient  amounts  of  data,  i.e. .  the  establishment  of  an 
ensemble  ("sample")  of  sufficient  size— to  provide  statistically  accurate 
decisions,  or  in  other  words,  to  yield  decisions  with  acceptably  small 
probabilities  (a,P)  of  error.  What  is  "acceptable,"  of  course,  will  d;.pend 
on  the  experimental  environment  and  the  accuracies  desired. 

To  summarize  then,  we  proceed  a^  follows  to  construct  a  statistical 

test: 
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A.  Select  the  hypothesis  (H^)  whose  validity  at  significance  level 
a  is  to  be  tested  (vs  some  alternative  i.e. ,  all  other  possibilities, 
for  example),  appropriate  to  the  experiment  tindertaken. 

B.  Select  an  appropriate  test  statistic,  Z(X) .  T^is  generally 
means  selecting  an  appropriate  type  of  test. 

C.  Choose  a,  and  therefore  (in  principle,  at  least)  obtain  the 
threshold  from  Eq.  (2.U).  This  usually  means  that  Wj^(z|h^)  is  known, 
or  approxiraable . 

D.  With  an  experimental  sample  =  Z(Xg^pj^g),  apply  the 

decision  rules: 

Accept  H  (at  significance  Level  a)  if  Z  ,  S  Zi  or 
o  sample  ot 

Reject  H  (at  significance  level  a)  if  Z  ,  >  Z^( and.*. accept  H, ). 

o  sample  ot  i 

Particular  care,  moreover,  must  be  taken  to  insure  (l),that  these  data, 
ifsample'  belong  to  a  valid  ensemble,  and  (2),  that  sufficient  replication  is 
made,  i.e. ,  the  sample  is  large  enough,  under  proper  ensemble  conditions,  to 
yield  acceptable  (i.e.,  "reasonable")  probabilities  of  correct  decisions. 
Usually,  only  W^(z(H^)  is  available,  so  the  only  control  on  the  test  of 
vs.Hj^  is  given  by  the  Type  I  erx'or  probability  a.  If  at  all  possible,  the 
attempt  to  determine  the  Type  II  error  probability  3  should  be  made;  this 
can  be  done  when  the  mechanism  governing  the  process  X  under  the  hypothesis 
is  known,  which,  unfortunatel.y,  is  not  a  frequent  experimental  situation, 
however . 

V/ith  the  above  in  mind,  let  us  proceed  to  the  program  sketched  in 
Sec .  1  earlier . 
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Ill .  Tests  for  Independence  of  the  Data  and  Validity  of  the 

Ensemble  (Homogeneity  and  Independence) 

As  we  have  mentioned  earlier  (Sec.  l)  a  collection  of  data  samples 
does  not  necessarily  constitute  an  ensemble  (of  finite  size)  which  may  be 
regarded  as  a  subset  of  the  infinite-member  ensemble  that  represents  the 
usual  theoretical  limit.  If  the  underlying  mechanism  is  cheuiging  in  the 
course  of  the  observation  we  cannot  expect  the  statistical  properties  of 
the  experimental  subset  to  approach  those  of  the  infinite  population,  under 
stable  conditions,  so  that  the  subset  itself  is  not  in  any  way  representative 
of  other  subsets,  obtained  subsequently,  and  so  on.  The  defining  statistical 
character  of  an  ensemble  is  that  it  have  reproducible  measures  or  properties- 
in  the  case  of  finite  samples,  these  should  converge  in  some  sense  to  the 
corresponding  measures  or  properties  characteristic  of  the  infinite  population 

Let  us  elaborate  further  and  discuss  the  construction  of  a  valid 
ensemble.  A  key  question  here  is  what  is  meant  by  an  ensemble  (Ref.  15).  As 
noted  above  in  Sec.  1, 

"An  ensemble  is  a  set  of  similarly  prepared  functions  (here  of  time, 
space,  or  both,  etc.)  to  which  can  be  assigned  a  probability  measure.” 

The  critical  point  here  is  that  the  ensemble  is  an  unordered  set,  in  that  the 
same  properties  (e .g. ,  statistics,  distribution,  etc.)  of  the  set  remain 
invariant  under  an  arbitrary  ordering.  Changing  the  order  of  an  ordered  set, 

1 .e. ,  disordering  it,  however,  implies  changing  the  mechanism  of  the  re¬ 
presentations  ,  and  that,  of  course,  destroys  the  original  set  properties. 

Thus,  if  is  the  ensemble  (j  =  1,...,M),  with  various  statistical 

properties,  such  as  the  mean  X,  mean-square  X  ,  p.d.,  W^(X) ,  etc.,  these 
remain  unchanged  (except  for  a  subset  of  measure  zero)  as  the  ordering  (or 
"indexing")  j  is  arbitrarily  changed,  e .g. ,  j  -•  j  randomly  or  otherwise. 

For  example,  if 


♦Adapted  from  ONR  notes  of  D.  Middleton. 
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—  1  ** 

X  =  r:  y  X(j)  =  (ensemble)  mean  of  X  ,  (3«l) 

"r-i 

then.  If  the  [X  J  form  a  true  ensemble,  we  must  have 
M  ,  M 

X^-^  ^  =  i  £  X^*’^  (j  ^  j'  arbitrarily),  (3.2) 

d'=l  J=1 

and  so  on  for  the  other  statistical  properties  of  the  set,  which  are 
accordingly  invariant  with  respect  to  the  index  (j). 

Ideally,  we  can  think  of  generating  an  ensemble  by  producing  the  set 
of  random  functions  simultaneously,  under  identical  conditions  and  identical 
statistical  mechanisms.  For  example,  an  ensemble  of  M  independent  "pings" 
from  a  sonar  producing  reverberation  in  a  medium  and/or  returns  from  a  target 
is  theoretically  obtained  by  introducing  an  identical,  single  ping  into 
each  of  M  independent  media,  each  of  which  itself  possesses  the  same  basic 
scattering  properties.  The  identical  pings  are  all  released  at  the  seme 
instant  from  the  M  distinct  sources.  Each  medium  also  has  a  receiver  at  some 
point  in  it,  and  each  receiver  is  identical  with  all  the  others  vis-a-vis 
location  and  processing  properties.  The  observed  reverberation  then  consists 
of  M  representations  X^^^(t),  X^^^(t),...,  X^^^(t),  tj^  <  t  <  t^^  +  T,  on  the 
same  interval  (t^,  t^  +  T) .  By  definition  of  the  manner  in  which  each  re¬ 
presentation  is  generated,  each  is  independent  of  the  other,  in  the  sense 
that  given  X^‘^^(t)  we  can  in  no  way  construct  X^^^(t)  (k  /  j)  from  it:  the 
reindom  mechanisms  producing  X^*^^  and  X^*^^  are  separate  and  unrelated.  From 
this  follows  the  required  invariance  of  ensemble  properties  with  respect  to 
ordering.  If,  however,  the  M  scatter  mechanisms  of  the  idealized  experiment 
above  are  not  independent  or  are  not  the  same,  ordering  will  depend  on 
coupling  and  mechanism,  and  we  no  longer  have  an  ensemble,  in  the  desired 
sense,  whose  properties  are  invariant  of  index. 


14 


Theoretically,  we  can  always  construct  (on  paper)  M  identical  ramdom 
mechanisms  and  operate  them  simultaneously.  Practically,  this  is  nearly 
always  impossible.  Instead,  we  must  take  our  data  sequentiaJ-ly  in  time, 
generating  one  long  record  which  under  certain  conditions  we  can  split  up 
into  independent  representations  which  then  can  constitute  eui  ensemble.  In 
place  of  our  example  above  we  now  have  a  single  source  of  pings  emd  a 
single  scatter  medium.  The  medium  is  "pinged"  at  different  times  t^j,  j=l, ..., 
M,  none  of  which  is  commensurable  with  another.  Sufficient  intervals 

-  t^j)  between  successive  pings  are  allowed,  so  that  eacn  reverberatory 
return  from  the  medium  dies  down  at  the  receiver  before  the  next  appears. 

Then,  we  take  the  single  record  of  M  consecutive  ping  returns  and  split  it 
up  into  M  separate  returns,  or  representations,  one  for  each  individual  ping. 
Each  representation  is  then  potentially  a  member  of  an  ensemble  (of  size  M) . 

If  each  repre  .tation  is  independent  and  the  underlying  scatter  mechanism  of 
the  medium  does  not  change  during  the  overall  time  (t^j^  -  >  we  then  have 

an  ensemble  in  the  statistics’  sense  defined  above.  In  more  recondite 
language,  we  require  that  the  mechanism  be  "ergodic,"  or  more  simply, 

"stable,"  with  independent  excitations.  Thus,  we  see  that  "independence" 
of  the  representations  is  closely  related  to  the  existence  of  the  ensemble, 
so  that  a  test  for  independence  of  the  representations  is  essential  to  a 
test  for  the  existence  of  the  ensemble,  as  is  a  test  of  "homogeneity,"  implying 
invariance  with  respect  to  ordering  of  the  members,  etc.;  there  should  be 
no  change  in  ensemble  properties  (cf.  (5*1) >  (5*2))  if  we  have  a  true 
ensemble,  as  the  ordering  (j)  is  changed. 

Similar  remarks  apply  to  a  target  enbedded  in  reverberation,  or  any 
background  noise.  Ideally,  we  would  prefer  that  position  and  aspect  of 
the  target  be  unchanging  in  the  course  of  time,  or  at  worst,  changing  in  a 
completely  predictable  way.  Physically,  however,  this  is  also  never 
strictly  the  case:  there  are  always  aspect  and  positional  variations  that 
sure  both  deterministic  and  entirely  rsindom,  and  which  are  not  predictable 
to  the  observer.  We  can  still  obtain  a  valid  ensemble,  for  target  plus 


*See  Sec.  1.6  of  Ref.  1  for  a  more  precise  definition. 
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background,  provided  again  that  the  mechemlsm  of  random  tsurget  variation 
remains  basically  unchanged  in  the  observation  period,  and  that  the 
deterministic  (i.e. ,  systematic)  effects  are  compensated  for.  For  instance, 
with  the  target  at  a  given  aspect  (and  fixed  transmitting  and  receiving 
platforms)  there  will  be  (small)  random  variations  in  the  target  returns, 
due  to  random  rotational  and  translational  target  movement.  If  the  aspect 
is  alowly  shifted  from  one  bearing  to  another  in  the  course  of  the  overall 
observation,  the  record  at  the  initial  bearing  will  obviously  be  different 
from  that  at  a  later  period  of  the  experiment.  Consequently,  whereas 
these  data  at  the  earlier  time  can  be  split  up  into  representations  that 
may  form  an  ensemble,  i.e. .  may  be  homogeneous  and  Independent  (if  the  rate  of 
change  of  aspect  is  slow  compared  to  ping-rate,  of  course),  and  similarly  for 
these  data  taken  at  later  aspect  periods,  the  various  sets  of  representations 
will  not  in  any  combination  constitute  a  valid  ensemble,  since  the  under¬ 
lying  mechanism  is  no  longer  stable,  or  "stationary,"  but  changed.  Tests 
of  homogeneity  also  provide  a  measure  of  the  size  of  the  ensemble  (and 
interval)  which  can  be  obtained  under  such  slowly  varying,  "unstable" 
conditions . 

With  the  above  in  mind,  we  must  now  cite  some  specific  tests  for 
the  existence  of  a  valid  ensemble  or,  equivalently,  for  the  homogeneity  and  in¬ 
dependence  of  a  data  set  (sample  ensemble).  Let  us  consider,  aga5n  as 
our  example,  a  set  of  reverberation  data,  containing  a  target  at  some  aspect, 
as  shown  in  Fig.  (3-1) •  This  set  is  obtained,  as  above,  by  repeated,  in¬ 
dependent  pinging  of  the  target  and  medium.  Our  basic  question  here  is, 

"Do  we  have  a  valid  ensemble?".  We  proceed  first  by  splitting  the  set  into 
two  subsets  and  then  testing  for  the  homogeneity  of  the  two  subsets.  By 
successively  interchanging  the  members  of  each  subset  and  repeating  the 
test,  we  can  then  establish  the  homogeneity  of  the  total  set.  (Other 
variations  of  this  procedure  are  briefly  indicated  below.) 
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FIGURE  3.1 

TWO  INDEPENDENT  SETS 

(s=1.2)OFMi,  M2  samples  OF  REVERBERATION  TRACES 
WITH  A  TARGET  (SKETCHES  OF  ENVELOPE  DATA) 


Lot  =  M  =  total  niimber  of  reverberation  (and  target)  records 

generated  in  an  experiment,  and  let  represent  the  number  in  a  pair  of 

sequences  s,,  (sample  size  M, ) ,  s_(sample  size  M~) .  Successive  pings  are 

1  it:  d 

produced  sufficiently  far  apart  in  time  that  there  is  no  overlap  of  returns 

from  ping  to  ping,  and  the  total  duration  of  the  experiment  may  be 

not  so  long  that  the  underlying  propagation  mechajasims  change.  Our  initial 

question  is:  "Is  the  set  s^^  drawn  from  the  same  population  as  the  set  s^ 

(and  vice  versa)?".  Now  we  form  a  grouping  into 

follows :  Let 


=  no.  of  samples  (in  s^  set)  where  X(t^)  falls  in  range  X^(t^), 


X.(t^)  +  AX; 


E  =  Mr 

i=l 


(5.3) 


=  no.  of  samples  (in  s^  set)  where  X(t^)  falls  in  range  X^(t^); 


X^(t_j^)  +  fix; 


Z  >‘l  '  Ms- 

i=l 


( 1 ) 

Each  group,  therefore,  has  a  set  of  probabilities  (p  , 

,  (1).  „  (2)  „  (2)  „  (2)  ^ 


Pi 


(1) 


ri 


5  Pi 


• } 


. ..,  p  ■  ')  associated  with  the  i 
^2 


1, 


1’ 


i  =  1, 


r^  groups  of  amplitude  levels.  Here  it  is  convenient  to  set  the 


number  of  levels  the  same  in  each  group,  so  that  r^  =  r^  =  r.  The  prc- 
r  1  ^ 

liabilities  (Z  p.  =  l)  are  not  known  here,  but  the  test  of  homogeneity  of 
x=l  ^ 

the  sequences  is  equivalent  now  to  establishing  the  hypothesis  (H^)  that 
these  probabilities  are  constants .  which  remain  the  same  (for  the  same 
interval  levels)  from  sec^u^T (s^)  to  sequence  (s^).  In  other  words,  the 
underlying  statistical  description,  as  given  by  the  various  p.d.'s  of  the 
(infinite)  ensemble,  remain  the  same  for  the  set  s^  as  for  the  set  s^. 


18 


We  are  now  ready  to  describe  a  number  of  tests  for  homogeneity  of  the 
(sample)  ensemble:  the  first  is  distribution-dependent,  while  the  second  and 
third  are  distribution-free  : 

2 

A.  Ax  “test  for  Homogeneity  of  the  Sample. 


It  is  shown  by  Cramer  (Ref.  lU)  that  the  test  statistic  for 
testing  the  homogeneity  of  the  two  sequences  (subsets),  that  comprise  the 
experimental  ensemble  s^^  ^  described  above  is 


■'sample 


=  X 


V2I 


i=l 


(3.1^) 


2  2 

where  x  is  distributed  with  a  x  “  of  r  -  1  degrees  of  freedom  (r  5  2). 

Specifically,  this  p.d.  is 

r“l  r-5 

Wj_(z|Ho)  =  [Wj_(x^|Ho)=]  {2^  r(^)}  e’^/^,  Z  >  0,  (5.5) 

(cf.  Ref.  2,  Eq.  30*1*2).  In  particular,  r  S  2  and  we  must  have  M^,  Mg  >  >  r 
for  the  sampling  distribution,  i.e. ,  ^q^^sampie^’  converge  in  probability 
to  the  infinite  sample  p.d.  W^(z|Hq)  above  (cf.  Secs.  50.1,  30.6,  Ref.  2). 

If  =  Mg  =  M/2,  which  is  often  a  convenient  "sizing"  for  comparison,  the 
test  statistic  (3*^)  becomes 


■'sample 


r  (v,  -  ^,)‘ 
=  y  — i - L- 

.  .  V,  +  \x. 

i=l  ‘  ’■ 


(5.6) 


with  r  5  2,  and  M/2  >  >  r  for  us  to  be  able  to  use  Eq.  (5 -5)  S’S  the  p.d.  of 
sample  Z.  We  next  select  our  significance  level,  a,  and  determine  the 
threshold  Z^  according  to 


*Here  by  distribution-dependent  we  mean  that  the  test  statistic,  Z,  obeys  a 
specified  form  of  probability  distribution,  whereas  in  the  "distribution-free" 
case,  Z  does  not  depend  on  a  specific  form  of  p.d. 
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a 


00 

P(Z  >  7.Jr)  =  J  W^(z(H^)dZ;  with  W^(Z(H^)  -  Eq.  (3.5), 


Z  =x 

a  '•  a 


(5.7) 


2 

for  r  degrees  of  freedom  from  the  Table  of  the  x  distribution  (Table  1, 

Appendix,  or  cf.  p.  '359>  Cramer,  Ref.  2,  for  example).  Then  we  determine 

Z  from  Eqs.  (3-^),  or  (3*6),  and  apply  the  test  (2.3): 

ssunpxc 


r 


decide  H  :  if  Z  ,  <  Z  :  the  two  sequences  came  from  the  same 

_ o  sample  a  ^ 


statistical  population,  i  .e . .  belong  to  the  same  dis¬ 


tribution,  or 


(3.8) 


4 


decide  if  S  Z^:  the  tvro  sequences  do  not  come  from  the 

same  population:  sequence  s^^  is  not  homogeneous  with 
sequence  s^* 


The  decision  then  has  a  (conditional)  probability  1  -  a  of  being  correct, 
Note:  This  test  is  a  first-order  test,  i.e. ,  only  the  first-order  p.d.'s 
of  are  involved  in  determining  the  levels  X^,  etc. 


Example ; 


As  an  example,  let  us  consider  the  hypothetical  case  of  a  set  of 
reverberation  pings,  presumably  independently  generated,  of  the  oxr'eriment 
described  above  and  il  ustrated  iu  Fig.  (3-1)  •  Let  M  =  200,  i.e.,  200  records 
with  1  ping  each  is  the  total  sample  ensemble  size,  and  let  us  consider  the 
fecvelopes)  X^'^^  of  the  retiirns  at  a  time  t^,  (t^e  T) .  v^e  divide  the  range 


of  amplitudes  of  into  10  intervals,  all  equal  except  for  the  tenth, 
which  ranges  from  9AX  to  <».  Then  =  M/2  =  100  and  r^  =  r^  =  r  =  10, 

if  we  divide  the  set  into  two  equal  parts  for  the  test  of  homogeneity. 

From  our  data  let  us  say  that  we  find  that 


=3;  ^2  ~  ~  '^5  ~  ^^6  ~  ~ 


vq  =  5;  =  3;  v^q  =  2, 


(3-9) 


=  1;  =  21;  \i  =  I7;  |ig  =  13;  u-  =  10; 


^8  “  ^10 


=  1. 


From  Eq.  (3*6)  our  test  statistic  is  computed  to  be 


2s«.ple  =  “  ‘t  -  ‘5.10) 

and  from  the  Table  1  (or  Ref.  2,  p.  559) >  for  r  -  1  *  9  deg  of  freedom  we  see 

that  «  16.92  at  the  5<  (a  =  O.O5)  significemce  level.  Evidently, 

<  =  16.92  here,  and  H  is  therefore  accepted:  the  two  sequences  are 

.  VJp  o 

homogeneous,  with  a  0.95  probability  of  this  being  so,  and  with  0,05  pro¬ 
bability  that  this  hypothesis  (H^)  is  falsely  rejected.  This  test  is  then 
repeated  for  the  different  ranges  (t^e  T) .  If  homogeneicy  prevails  (at 
this  a-signif  j  cance  level)  we  can  conclude  taat  these  data  everywhere  in 
the  range  interval  considered  effectively  belong  to  a  valid  ensemble.  If 
there  are  ranges  (t. ')  where  H  is  rejected,  we  conclude  tentatively  that 
different  random  mechanisms  are  at  work  in  the  two  subsets,  at  least  at 
these  ranges. 


Of  course,  we  could  raise  our  threshold  Z^  by  decreasing  the  false 
rejection  probability  a,  and  tighten  the  test  with  respect  to  H^.  However, 
at  the  same  time  we  would  increase  3,  the  (conditional)  probability  of 
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falsely  deciding  when  "iuhoraogeneity"  of  s^  ^  true.  Since 
we  do  not  here  know  the  limiting  distribution  (let  alone  the  sample  dis¬ 
tribution)  of  the  test  statistic  Z  under  the  alternative  we  cannot  use 
El*  (2.5)  to  determine  p.  Clearly,  one  reason  for  looking  for  "best"  tests, 
in  the  sense  at  leeist  of  minimizing  3  (or  maximizing  the  power,  1-3),  is 
that  for  a  given  control  or  significance  level,  a,  we  can  hope  to  keep  3 
acceptably  small.  In  any  case,  choosing  a  too  small  penalizes  us  by  a 
correspondingly  large  3!  falsely  deciding  when  it  really  is  not  true. 

As  noted  in  Sec.  2,  the  threshold  x'eally  is  the  ratio  of  two  other 
sub,1ective  factors:  \/p  -  ratio  of  cost  or  value  assignments  to  the  possible 
decisions,  to  the  ratio  of  a  priori  probabilities  that  the  data  at  hand  do 
or  do  not  belong  to  H^,  cf.  (2.10).  Selecting  a  threshold,  accord¬ 
ingly,  involves  elements  outside  the  physical  data  themselves  and  includes 
the  general  significance  and  "value"  of  the  experiment  and  its  consequences. 
(Statistical  Decision  Theory  (SDT)  includes  such  elements,  in  a  broader 
formulation  of  the  decision  process  in  the  face  of  uncertainty  (Ref.  I5, 

Chaps.  1,  2,  and  5)  and  (Ref.  1,  Chaps.  18  and  19)-) 

So  far,  besides  independence  of  the  representations, we  have  assumed 
that  each  data  group  s^,  s^  is  itself  a  valid  subensemble  and  what  we  are 
testing  for  is  whether  or  not  the  combined  data  belong  to  a  single  statistical 
population  (at  each  of  the  rsmges  t^cT) .  This  is  really  a  test  of  the 
"stability"  of  the  ensemble,  here  at  times  t^.  In  our  example  above  we  have 
shown  that  there  is  no  apparent  change  of  statistical  mechanism  between  the 
time  when  the  first  representation  was  obtained,  through  the  time  of  the  last, 
so  that  if  subsets  1  and  2  are  themselves  valid  subensembles,  the  complete 
data  set  is  likewise.  However,  to  establish  that  each  subset  is  a  valid 
subensemble  we  must  be  able  to  interchange  representations  without  damaging 
ensemble  properties  —  i  .e. ,  we  must  have  on  unordered  set,  and  this  in  turn 
.'mplies,  basically,  independence  of  the  mechanisms  generating  each  representation. 

There  is  no  unique  way  of  establishing  this  fact.  One  possibility 
is  to  d'vide  up  the  complete  data  into  now  four  equal  parts  and  test  these 
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against  each  other  pair  by  pair  (or)  jointly,  by  an  extension  of  Cramer 
(Ref.  2,  Sec.  30*6,  for  S  =  4,  etc.).  If  homogeneous,  continue,  splitting 
each  quarter  in  half,  etc.  The  trouble  with  carrying  this  too  far  (e.g. , 
when  the  number  of  degrees  of  freedom  (r)  becomes  comparable  to  or  larger 
than  the  number  of  representations  in  the  subgroup)  is  that  we  begin  to  get 
significant  results  too  often:  there  are  not  enough  data  to  provide  adequate 
sample  sizes. 

Ano'her  possibility  is  to  exchange  every  odd-indexed  representation  in 
Sj^  with  its  counterpart  in  and  test  for  homogeneity  using  the  above 
procedure.  Then,  replace  these  elements  and  exchange  all  even-numbered 
representations  and  repeat  the  test.  If  is  rejected  in  one  or  both 
tests  we  strongly  suspect  that  the  subensembles  are  not  valid,  i.e. ,  do  not 
consist  of  independent  (i.e. ,  unordered)  representations.  (More  refined 
(and  complicated)  tests  of  independence  exist;  these  involve  calculation  of 
"contingency,"  cf.  Ref.  2,  Sec.  30.5,  but  will  not  be  discussed  here 
further.)  Usually,  the  above  may  suffice,  at  least  as  an  initial  approach. 

Of  course,  if  there  are  cyclical  mechanisms  with  several  cycles  at  least 
in  a  subgroup,  these  will  not  be  revealed,  and  a  valid  ensenble  may  be 
decided,  when  such  is  not  the  case.  (See  C.  below,  on  the  "runs  test.") 

Again,  we  must  refer  as  much  as  we  can  to  our  knowledge  of  the  basic  physical 
mechanisms  that  govern  these  data,  to  assist  us  in  choosing  our  tests  euid 
interpreting  the  results. 

B.  A  Nonpareunetric  Test  for  Homogeneity  of  the  Data;  The  Kolmogorov- 

Smlrnov  Test: 

This  is  the  Kolmogorov-Smirnov  test  (Refs.  l6  and  17,  and  in 

particular,  pp.  47-52;  127-136  of  Ref.  8),  which  has  the  considerable 

2 

advantage  over  the  y  -test  above  in  that  it  is  distribution-free,  i.e., 
it  does  not  depend  on  the  (infinite)  population  statistics,  but  only  on 
those  of  the  sample.  Furthermore,  it  is  usually  more  powerful  (i-e. , 
given  larger  values  of  1-P)  than  does  the  \  .  (Ref.  8,  p.  127,  et  seq.) 
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For  the  basic  problem  here  of  testing  the  homogeneity  of  the 
data  set,  we  again  divide  the  sample  (of  size  M)  into  two  sets  of  representation, 
of  size  M^,  with  =  M.  The  test  statistic  now  is  (Ref.  8,  p.  127, 

et  seq.)  : 


'sample  “  -®<  X(t^)  <  ®  I  ‘  I 


(3.11) 


where  cumulative  experimental  distributions  of 

X,  taken  over  the  sets  j  =  1,...,M^,  j  -  +  1, . . . ,  =  M,  at  time 

t,  ,  when  the  X^  are  arranged  in  order  of  (ascending)  magnitude  in  each 
subset  The  test  based  on  (5.II)  is  a  two-tailed  test.  It  is 

sensitive  to  any  kind  of  difference  in  the  distributions  from  which  the  two 
subsets  are  drawn--such  as  location,  dispersion,  etc.,  as  well  as  the  form 
of  the  distributions  themselves. 

The  test  of  whether  the  subensemblei  (Sj^,  s^)  are  from  the  same 
population  is  then,  as  before. 


decide  the  two  sets  are  homogeneous,  i  .e . ,  belong  to  the  same 


distribution,  if  Z  ,  ?  Z. 

’  s  ample  a' 


(3.12) 


1  decide  H^:  the  two  sets  are  not  homogeneous,  i  .e . ,  belong  to  different 


distributions,  if  Z  ^  >  Z. 

’  sample  a' 


where  now,  however,  the  threshold  Z^  is  given  by 

Ca 


X 

a  ■  ‘ 


(3.13r) 


2k 


and  is  determined  by 


a  =  1  “Z 


2  2 

-  ,  a 

(-1)  e 


k=-<o 


(3.13b) 


which  relates  the  significance  level,  a,  to  the  threshold  parameter  \  . 

sX 

When  =  M/2,  (3.13a)  simplifies  to  =  2\^yiT.  To  carry  out  an 

actual  test,  with  chosen  a,  we  must  first  obtain  (These  values  are 

provided  for  in  Table  3,  Appendix;  \  is  the  coefficient  of 
,  i  ® 

(M^+M2)(M^,  M^)^  in  Table  3.  Table  2  is  used  in  the  small  sample  case 

when  M,  =  S  UO,  and  the  test  statistic  is  K  =  M,  Z  ,  ,  i.e.,  the 
12’  1  sample’  - ’ 

maximum  numerator  (3.II),  and  the  threshold  is  replaced  by  K^. ) 

One-tailed  tests  can  also  be  constructed.  Thet;e  use  the  test 

statistic 


^sample  *  -«d  <  x(t^)  <  »  ’  (3.ll+a) 

and  are  employed  to  decide  whether  or  not  the  values  (magnitudes)  of  the 
population  (i.e. ,  the  theoretical  ensemble)  from  which  one  of  the  subsets 
was  drawn  are  statistically  larger  than  the  magnitudes  of  the  population 
from  which  the  other  subset  is  taken.  For  small  samples  (M^  =  S  Uo) 
critical  thresholds  are  given  by  Table  2  (Appendix),  where  now  the 
test  statistic  is  iiiodified  to  K  .  =  M,  Z  .  =  M  ■  Eq.  (3.lUa). 

For  large  samples  (M^,  ^  ^+0),  the  test  statistic  is  shown  to  be  (Ref.  8, 

P.  151) 


Z  ' 


sample 


=  hZ" 


1  d 

sample 


(3.lUb) 


where  Z  obeys  the  y^-distribution  of  (3.5)  '••'ith  2  degrees  of  freedom; 

the  significance  level  a  is  determined  by  (5.7)  >  and  the  test  itself  is 
given  by  (3.8).  (See  Table  1,  Appendix.) 
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From  this  point  on  we  test  for  homogeneity  and  validity  of  the 

sanple  ensembles  precisely  as  in  the  preceding  Case  A  above,  repeating  for 

each  time  (range)  t^  in  the  records.  In  practice,  we  would  choose  only  a 

few  such  ranges  usually,  unless  we  had  reason  to  suspect  possible  variations 

in  the  statistical  mechanisms  for  some  local  range  interval.  Over  the 

target  cur  selection  of  t^'s  would  be  denser,  to  examine  the  target  return 

for  any  level  features .  Whereas  the  amount  of  computation  for  the  Kolmogorov- 

2 

Smirnov  test  is  usually  larger  than  the  x  test,  the  effectiveness  of  the 
former  is  less  sensitive  to  the  true  distribution  involved  and  is  more  powerful. 
(Again,  unless  we  know  the  distribution  of  the  test  statistic  under 
we  cannot  determine  fully  how  accurate  our  decisions  are.) 


Example ; 


Let  us  again  return  to  the  data  of  our  reverberation  example, 
discussed  in  A,  above.  The  are  the  magnitudes  of  the  reverberation 
envelopes  (cf.,  Fig.  (3.1)),  which  for  the  unordered  data  set  are 


X^^^(t^)  =  5;  X^^\t^)  -  2;....;  X^^^(t^)  =  7;  — ;  X  ^  (t^) 


(Mj 


=  U 


smd 


(Sj^  sequence  of  representations) 


(J--M,+1)  (M,+2)  (M  +M  ) 

X  (t^)  =  8;  X  (t^)  =  5;....;  X 


(s^  sequence  of  representations). 


(5.15a) 


(t^) 


=  6. 


These  become,  on  being  ordered, 

(Si);  x[^^  2;  X^  X^^^  =  5; - ;  I6; 

(M  +2)  (M  +M  )  (M  +1) 

(s^)  I  X^....*  X^  =6;....;  X^  =  8;, 


(3.15b) 
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Next,  we  determine  the  cumulative  distributions  Sj^  noting 

that  there  may  be  more  than  one  sample  member  having  the  same  value.  This 

is  done  by  adding  increments  to  the  preceding  value  of  Sj^^, 

S,,  ,  at  S.  ,,  respectively  for  the  two  data  subsets.  (Again,  v. ,  p.  are 

number  of  ordered  X^  having  the  same  values,  cf.,  (5-5)  •)  From  these 

distributions  we  take  the  largest  difference,  according  to  (j.ll)  or  (3*l*+)j 

and  this  is  the  desired  test  statistic  Z  ,  .  This  is  shown  in  Fig.  (3.2) 

sample 

and  in  Table  (3.l),  0-ncluding  first  and  second  sample  moments)  for  our 
reverberation  example. 


Here  =  100  and  again  we  choose  the  significance  level 

a  =  0.05.  From  Table  3  (Appendix)  applied  to  (3.12),  we  get 

^0.05  '  2\).05/'^  =  '^2  •  1“'^  N).05  '  x.0.05  “ 


and  from  Table  3.1  and  Fig.  3*2  we  see  that  the  test  statistic  (3.II) 
here  becomes  (for  the  present  two-tailed  test) 


Z 


sample 


0.07 


(3.16b) 


so  that  Z  ,  <  Z^  and  therefore  we  conclude  from  (3.12)  that  (at 

sample  O.O5'  ' 

the  0.05  significance  level)  the  two  subensembles  (M^,  M^)  are  homogeneous, 
1  .e. ,  belong  to  the  sajne  distribution. 


We  can  also  determine  whether  the  values  of  X  belonging  to 
subset  1  are  statistically  compeurable  to  those  belonging  to  subset  2  by 
using  the  one-tailed  test  statistic  (3.1^),  which  for  the  present  large 
samples  is  modified  to  (3.1^a),  viz 

2  2 

Z'gample  "  =  O.98  -  (2  deg.  of  freedom)  .  (3.1?) 
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TABLE  (3.1) 


i 

range 

No.  of  magnitudes 
of  X^in  interval  AX^ 

>1 

^2 

\  -  % 

B 

0-1 

v^-3 

Ui=  1 

0.03 

0.01 

0.02 

1-2 

1 

6 

O.OU 

mStm 

-0.03 

2-3 

17 

16 

0.21 

-0.02 

3  - 

16 

21 

0.37 

■m 

-0.07 

5 

1*  -  5 

28 

17 

0.65 

0.6l 

O.OU 

6 

5  -  6 

lU 

13 

0.79 

■SB 

0.05 

7 

6-7 

11 

10 

0.90 

■SB 

0.06 

8 

7-8 

5 

8 

0.95 

BB 

0.03 

9 

8-9 

3 

7 

0.98 

-0.01 

P  •  00 

2 

1 

1.00 

1.00 

0.00 

«  100  ■  100 


0^  =  20.3  ;  Oy  =  21.6 
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2  3 

From  Table  1  we  see  that  X  q  Qrjo^f  ""  5*99,  while  P(Z  >  ^  o.05-2df  ^ 

0.62  (>  a  =  0.0^)).  Thus,  we  conclude  that  "no  statistical  difference 
in  mar:nitudes"--applies ,  at  0(  =  O.O5,  which  is,  as  expected,  consistent  with 
the  result  of  the  two-tailed  test,  that  the  two  subensembles  are  homogeneous. 

C .  A  Test  of  Sample  Randomness  and  Validity  of  the  Ensemble: 

One-Sample  Runs  Test 

A  key  condition  that  the  data  sample  constitutes  a  valid 
(experimental)  ensemble  is  that  the  member  representation  be  statistically 
independent,  as  well  as  havin.c  the  same  underlying  statistical  mechanism. 

(See  the  remarks  at  the  begin.ning  of  Sec.  3,  above.)  A  useful  test  for 
establishing  this  fact,  viz.  the  "randomness"  of  the  sample,*  in  addition  to 
what  is  known  about  the  physical  mechanism  producing,  the  random  phenomenon — 
e.g. ,  ping-rate  in  our  specific  example,  is  the  so-called  "runs  test"  (cf. 

Ref.  8,  pp.  52-58),  which  uses  the  order  or  sequence  in  which  the  original 
data  are  obtained. 

This  test  is  based  on  the  number  of  runs  the  data  sample  exhibits, 
where  a  run  is  defined  as  a  succession  of  identical  symbols  (numbers,  +  or  - 
signs,  etc.)  which  are  preceded  and  followed  by  different  symbols  (or  none 
at  all).  The  idea  behind  a  test  using  runs  is  as  follows:  the  total  number 
of  runs  in  a  sample  ensemble  of  any  specified  size  gives  an  indication  of 
whether  or  not  the  sample  is  random.  For  very  few  runs,  some  causal  mechanism 
producing  a  time-trend  or  data  bunching  indicating  a  lack  of  element  in¬ 
dependence  is  suggested,  whereas  if  an  excessive  number  of  runs  occurs,  some 
systematic  cyclic,  short-period  mechanisms  appears  to  be  operating.  In 
either  instance,  independence  of  the  representations  does  not  seem  to  hold 
and  the  sample  is  said  not  to  be  random  (at  some  level  of  significance, 
a,  of  course,  which  is  selected  for  the  actual  tests)  .  The  importance  of 
a  test  based  on  runs  is  the  fact  that  the  order  in  which  the  events  occur 


*In  conventional  statistical  language,  the  sample  values  are  said  to  be  "random. 
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is  specifically  used,  unlike  A  and  B  above,  where  such  information  is 

2 

neplected  in  the  testint^.  Thus,  it  is  quite  possible  that  a  y  or 

Kolmogorov-Smirnov  test  might  indicate  acceptance  of  H^,  while  a  runs 

test  would  indicate  an  inhoraogeneity  in  the  statistical  mechanism,  i .e . , 

rejection  of  H  . 

0 

To  set  up  a  runs  test  we  select  two  attributes  of  the  sample, 

which  appear  in  sequence.  For  example,  one  attribute  might  be  all  values 

of  an  observed  magnitude  that  are  positive,  the  other,  all  negative  values. 

In  a  sample  with  M  representations,  let  be  the  number  having  attribute 

No.  1  (positive  values,  say),  and  the  number  having  attribute  No.  2 

(negative  values),  e .g. ,  M  =  Next,  observe  the  order  in  which 

these  attributes  occur  in  the  original  data  and  count  the  number  of  runs, 

R.  Thus,  the  test  statistic  is  Z  ,  =  R  here.  We  distinguish  two  cases: 

samplj 

Small  Samples  [M^,  <  10]: 

decide  H  ,  if  R_  ^  <  R  <  R-^  i.e.,  decide  that  the  sample  is 

random  -  the  elements  are  independent  and  unordered. 


decide  H, ,  if  R  <  R,.  ^,or  R  >  R-^  .  i.e.,  decide  that  the 

1’  I-a  Il-a  — 


elements  are  not  independent,  but  are  ordered  in  some  fashion. 


If  R  <  R^  ,  there  are  two  few  runs,  while  for  R  >  R-.^  there  are  too  many. 
I-a’  ’  II -a 

Here  R^.  ^  is  given  in  the  body  of  Table  t  and  R^^  ^  in  Table  5>  for  'the 
a  =  0.0‘5  significance  level. 


Large  Samples  [Mj^,  5  20]:  Here  the  sampling  p.d.  of  R  is 

essentially  normal,  with  mean  and  variance 


^1^2  .  .  .  2 
^  M^+M^  R 


2M^M2(2M^M2-M^-M2) 

(Mj^+Mp)^  (M^+M^-l) 


(5.18) 
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The  test  statistic  is  taken  to  be  the  standardized  normal  random  variable 


sample 


(3.19) 


emd  the  test  itself  is  two-tailed,  specifically: 

if  |Z  ,  I  <  |Z  I ,  we  conclude  that  the  sample  is 
'  sample'  'a” 

randomly  (i.e. ,  independently)  generated,  with  an 
invariant  statistical  mechanism, 

(3.20) 

if  |z  ,  I  >  Iz  I,  where  we  conclude  that  the 
'  sample'  '  a’’ 

sample  is  not  independently  generated  and  has  a 
varying  statistical  mechanism. 

In  short,  the  runs  test  provides  an  alternative  and  nonparametric  approach 
to  determining  whether  or  not  the  sample  ensemble  is  a  valid  one,  i.e., 
whether  it  can  be  regarded  as  belonging  to  a  single  random  process,  with 
specifyable  measures.  Here  we  have 


V  or 


decide  H  : 
o 


decide  : 


V. 


p(|z|  >  |zj)  =  O'  ,  (3.21) 

and  P/2  is  given  in  Table  6  (Appendix)  for  this  large-sample  case;  (the 
probability  P  is  twice  that  shown  in  Table  6  because  we  are  using  a  two- 
tailed  test) . 

/ 

/ 


/Let  us  now  apply  the  runs  test  to  the  reverberation  example  of 
the  preceding  portions  of  this  report,  keeping  as  before  a  significance 
level  a  =  0.05,  First,  we  note  that  our  data  [cf. ,  Fig.  (3.1)  and 
Table  (3.I)]  refer  to  envelope  values,  which  are  always  positive.  To  set 
up  a  meaningful  runs  test  let  us  use  as  our  sample  attributes  the  positive 
and  negative  differences  between  successive  envelope  magnitudes,  taken 
in  independent  pairs  -  X^,  A  =  X^^^  -  X^^^^  , 


Example 
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and  :ln  particular,  the  pluses  and  minuses  so  generated  in  the  sequence  of 
M  =  100  pairs  of  representations  (taken  at  time  t  =  t^,  as  before),  a 
typical  section  of  which,  let  us  say,  looks  like 


----  ++  -  ++  +++++  —  +  — -  +++++++  - 


FIGURE  (3.3) 

A  portion  of  the  sequence  of  +'s  and  -'s  (signs  of 
the  envelope  differences  of  the  reverberation 
sample  ensemble  of  Fig.  (3-1 )  at  t^.  (Each  run  is 
underlined. ) 


Let  us  also  say  that  in  the  sample  of  M  =  100  representation  (and  « . 
(+  or  -)  values  of  ;  no  zeros)  we  find  that  there  are  =  5^ 
(+'s)  and  Mg  =  (-'s),  and  R  =  58  runs.  A;  plyinn  (3-18)  through 

(3.20)  we  obtain 


100 


Op  =  4.9^; 


z 


sample 


b8  -  p0.6| 
"X95 


1.49 


(3.22) 


so  that  from  Table  6  we  see  that  P(|z|  >  "  2*  (O  O681)  =  O.1362, 

which  exceeds  a  =  O.O5.  Accordingly,  we  retain  the  hypoti  >sis  and 
conclude  that  our  sample  ensemble  is  a  random  sample,  from  a  single 
population. 


In  the  nreceding  three  cases  ^A  through  C)  we  have  shown  how 
a  variety  of  useful  statistical  tests  may  be  used  to  establish  the  validity 
of  the  sample  ensemble.  The  Kolmogorov-Smi rnov  test,  which  is  applicable  in 
all  cases  where  the  population  distribution  is  continuous  (as  is  certainly 
the  case  in  our  physical  applications),  is  usually  the  most  powerful. 

(For  the  runs  test  no  meaningful  stater urt  as  to  its  power  can  be  made, 
apart  from  the  possible  context  of  a  specific  problem.) 
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TV .  Tests  of  "Goodness-of-Fit 

Here  we  wish  to  test  the  hypothesis  (H^)  that  our  sample  ensemble 
belongs  to  some  specific  distribution;  (the  alternative  (H^)  is  any  other 
distribution).  In  other  words,  we  are  testing  for  a  "fit"  of  our  data  to 
the  postulated  distribution.  Again,  we  may  distinguish  both  parametric 
and  nonparametric  tests,  like  A  vs.  B,  C  in  Sec.  5  above.  We  begin  with 

2 

A.  A  X  -test  of  "Goodness-of-Fit;” 

In  this  case  we  have  M  independent*  data  samples  {X^'^^(t^)}, 
j  =  1,....  M,  say,  of  reverberation,  at  range  t^.  We  wish  to  determine 
whether  or  not  the  X's  here  belong  to  some  a  priori  specified  distribution, 
.  Rayleigh,  Hoyt,  etc.,  (Ref.  9),  where  the  X's  are  the  envelopes 
of  narrow-band  reverberation.  We  precede  as  follows;  cf.  Sec.  30.1, 

Ref.  2: 

1.  Divide  the  range  of  amplitudes  into  r  (equal)  intervals 
(except  for  the  r^*^) : 

X^;  X^  +  AX  =  X^;  X^  =  Xg  +  AX,  etc.,  where  X^^^^  =  +  AX; 

2.  Let  v/  =  number  of  values  of  the  set  of  M  sample  values 
having  X's  in  the  range  X^,  X.  +  AX  (as  in  (5.5)  above) 

V.'  M  ;  (Ij.l) 

i 

3-  Let  p^  =  probability  that  X^  falls  in  the  interval  X^, 

Xj^  +  AX,  determined  from  the  postulated  p.d.  against  which  we  are  testing, 

*'/fe  have  established  that  our  data  is  a  valid  sample  ensemble  by  the 
methods  of  Sec.  5. 


the  sample  for  "fit."  These  are  given  by* 

p.  =  W^(X.)  AX  ,  (U.2) 

where  Wj^(X^)  is  the  postulated  p.d.,  and  the  amplitude  interval.  AX  is 
chosen  according  to 


rax  = 


(‘•■5) 


where  X^^^  are  the  upper  and  lower  limits  on  the  range  of  values  of 

X  in  our  sample  set. 

Now,  it  can  be  shown  that  (cf.,  Sec.  30.1,  Ref.  2)  the  test 
statistic  here  is 


^sample  "  ^sample  /Mp^  , 


i=l 


and  that  as  M-w,  ~  density  with 


sample 


r-1  degrees  of  freedom.  Again,  a  is  determined  by  (3-7)>  ^or  this  Z 
(5s  Z,  cf.,  (i+.i*),  provided  M  >  >  r,  (rS2).  The  test  is,  accordingly, 

r 

decide  the  sample  ensemble  belongs  to  a  population  with 


or 


p.d.  W,  (X),  if  Z  .  S  Z  . 
^  1'  "  sample  a 


(1^.5) 


decide  the  sample  ensemble  belongs  to  a  population  with 


a  p.d.  W,(X),  If  >  Z„. 


■*The  X's  here  can  be  standardized  variables,  c.g.,  X,  =  (Y,  -  Y  )/ov  ,  where 

i  1  exp  '  lexp 

^exp>  ^  experimental  means  and  variances  of  the  original,  un- 

standardlzed  data  {y^^}.  Or,  equivalently,  the  X^'s  are  nonstandardized, 
and  we  use  the  experimental  means  and  variances  in  the  a  priori  p.d. 
as  above. 


35 


This  is,  of  course,  for  range  t^»  We  must  repeat  for  various  ranges  t^cT 
in  order  to  establish  with  more  reasonable  certainty  that  the  reverberation 
has  the  same  first-order  p.d.  as  that  postulated.  With  a  target  present 
(at  ranges  t^et^  +  AT,  say),cf.  Fig.  (3.1),  we  would  expect  a  non-fit 
to  the  postulated  d.d.  of  reverberation  alone,  even  when  W^(X)  for  this 
is  not  correctly  chosen.  Again,  from  the  structure  of  the  physical  model 
of  reverberation  (Ref.  9),  and  possible  models  of  target,  we  should  often 
be  able  to  make  a  good  guess  at  the  form  of  W^(X) . 

Example : 

Let  us  use  the  data  of  our  reverberation  example  (cf.,  Table  (3-l) 
earlier) to  test  whether  the  observed  envelopes  obey  a  Rayleigh  distribution, 


e-g-. 


«,(X|H  ) 

2 


-x^/x^ 


,  X  >  0 


(i4.6) 


Here  M  =  200;  and  ■*  r  =  10,  (Table  (3.I)).  The  p^  are  determined 

from  (U,6)  and  (1^.2).  However,  we  do  not  know  the  true  ensemble  value  of 
2 

X  ,  so  we  are  forced  to  use  an  estimate,  namely  the  sample  mean-square 


? 

A 


<='De 


Wexp  * 


=  20.71,  (cf..  Table  (3.I)).  (U.7) 


Consequently,  our  estimated  p^'s  are  obtained  from 
2X.  -X?/20.71 

'  ;(X^  =  0;X,  =  1,  X5  =  2,  etc.), 


(i^.B) 


and  arc  s;  ecif  ically,  with  the  accompanying  values  of  v^: 


p*  =  0.00; 

p*  =  0.095;  V2'  =  7 

p*  =  0.161;  v'  =  55 

p*  =  0.138;  vil  =  57 
p*  -  0.178;  V,'  -  h 


pj  =  O.lUU;  =  27 

p*  =  0.102;  =  21 

Pp  "  0»06^}  vq  =  15 

=  0.055;  =  10 

Bio  "  0.016;  V^Q  -  5 


(l-.8a) 


Applying  (l+.8a)  to  Eq.  (U.4)  gives  us  (on  discarding  the  values  at  i  =  1, 

=  0) 

"sample  *  =  •?  ,  (U.9) 

which  with  9  =  (r-l)  degrees  of  freedom,  gives  P(Z  >  =  0.27  from 

Table  1  ( Appendix]^  which  is  certainly  larger  than  our  chosen  significance 
level  a  =  O.O5.  Consequently,  we  accept  the  observed  data  do  fit 
a  Rayliegh  distribution  (with  evperimental  mean-square  20.71)- 

B.  A  Nonparmetric  Test  for  “Gtoodness-of-Fit"  (Kolmogorov-Smirnov  Test) ; 

This,  again,  is  a  Kolmogorov-Smirnov  two-tailed  test  (Ref.  8, 
pp.  i+7-51  and  Refs.  I6  and  I7),  constructed  as  follows:  the  test  statistic 
is  now 


Z 


sample 


max 

-®  <  X.(t^)<  ® 


|F(X(t^))  -  Sj^(X(t^))| 


(‘^.10) 


where  F  is  the  theoretical  (or  postulated)  cumulative  (first-order)  dis¬ 
tribution  of  the  ordered  X.(t^),  cf.,  Eq.  (5.15),  against  which  we  are 
testing  the  experimental  cumulative  distribution  Sj^(X_^) .  Thus, 

h 

F(Xj)  =/  Kj(X)  dX  ,  (1.11) 

«0B 

and  we  can  use  standardized  variables  for  X,  or  apply  the  same  experimental 
means  and  variances  to  the  e'qjlicit  structuring  of  F^^  through  Wj^(X) ,  (U.ll), 
(cf.,  the  footnote  preceding  Eq.  (U.2)).  The  threshold  at  level  a  is  now 
given  by 

X  (‘*•12) 


where  is  determined  by  (5.13b)  and  Table  5*  Thus,  the  test  takes  the 
largest  of  the  differences  |F-S„|  and  compares  it  to  the  threshold  Z^  in 
the  usual  way,  cf.,  (3.12).  (Figure  (5*2)  applies  here  with  s^^  replaced 
by  F  and  M  by  M^.)  The  test  is  repeated  for  the  different  rauiges,  t^, 
of  interest,  including  those  about  the  target. 
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Example : 


For  the  reverberation  example  treated  in  A  above  we  find  from 
(^4.6),  (*4.7)  in  that 


-X7/20.71 

F(X.)  1  -  e  "  ,  (Xj^  =  0;  X2  =  1;  X.  =  i-l) , 


(i+.l?) 


where  eigain  we  have  replaced  the  unknown  value  of  X  by  the  sample  estimate 
,  (l^!.7).  [This  procedure  is  an  approximation,  of  course,  vis-a-vis 
the  case  for  which  Table  7  is  calculated  under  the  assumption  of  F(X) 
completely  known,  including  its  statistical  parameters.  However,  there  is 
evidence  that  when  the  K-S  test  is  applied,  with  estimated  parameters  of 
the  distribution  used  in  place  of  the  (unknown)  true  values,  the  employment 
of  Table  7  will  lead  to  a  conservative  test — i.e.,  when  Z  ,  >  Z.  the 

hypothesis  may  be  even  more  safely  rejected  than  in  the  case  of  F 
completely  srecified  (and  test  at  level  a).]  Equation  (^.15)  for  our 
example  is  shown  in  Fig.  (3.2),  from  which  we  obtain  for  (I4.IO) 


^sample  =  M,  = 


100 


and  M  =  Mp  for  purposes  of  numerical  illustration.  [Of  course,  a  better 

test  would  be  to  use  all  data  M  =  M,+M„  =  200,  and  obtain  Z  .  for  M  =  200. 

1  2  '  sample 

The  r resent  case  is  chosen  simply  for  illustration.]  From  Table  7  we 
see  that  at  the  a  =  O.O5  level  of  significance, 


V05  “  ^  =  0.09  ,  (I..15: 

so  that  we  retain  and  accordingly  say  (for  this  significance  level)  that 

our  data  fit  the  Rayliegh  distribution  (U.15).  This  is  consistant  with  our 

2 

conclusion  under  A  above,  based  on  the  x  -test. 


C .  A  New  Nonparametric  Test  for  the  Normal  Distribution  (Ref.  l8) ; 

In  a  great  many  applications  it  is  reasonable,  for  physical 
reasons,  to  expect  normal,  or  gaussian  statistics — for  instance,  in  the 
case  of  ambient  sea  noise  (Ref.  I7) ,  and  in  our  example  discussed  above, 
where  the  (instantaneous)  reverberation  amplitudes  (as  distinct  from  the 
envelope  magnitudes)  will  generally  obey  a  normal  distribution  if  the  number 
of  independent  illuminated  scatterers  is  sufficiently  large  (Ref.  9,  Sec.  10.1 
2).  Accordingly,  it  is  practically  very  important  to  be  able  to  handle 
the  gaussian  cases  in  as  powerful  feishion  as  possible.  Now,  while  both 
the  X  and  Kolmogorov-Smirnov  (K-S)  tests  are  applicable  to  the  problem 
of  testing  whether  or  not  the  sample  ensemble  belongs  to  a  normal  process, 
at  least  with  respect  to  the  first-order  p.d.  W^(W;  t^) ,  a  recent  test,  called 
the  W-test  for  normality  (Ref.  I8),  has  been  derived  that  is  particularly 
powerful  (more  so  than  either  the  x  and  K-S  test)  and  is  quite  simple 
to  calculate.  The  test  statistic  here  is 

Z  ,  =  W  -  =  b^/S^  ,  (i^.l6) 

sample  sample  '  ’ 

where,  if  we  have  M  sample  values  X^^^(t, ),  j  =  1,...,  M,  before,  we 

( i)  ^ 

order  the  into  an  ordered  sample  set,  and  compute 

M  N 

s2  .y  (x^j).  (x^.  ,  (U.I7) 

l»l 

where  (x)  =  sample  mean,  and  we  obtain  b  from 

M/2 

(M  even):  b  *  h'> 

i=l 

(Mold):  b  =  e„(J(„.Xj)  .  (U.ieb) 

3  2  2 

where  the  a^  are  obtaned  from  Table  8  (Table  5  of  Ref.  18) .  Next,  we  compute 
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"sample  through  (lj.l8b)  and  use  Table  9  (Table  6 

of  Ref.  l8)  to  find  for  various  values  of  a.  Table  9  gives  values 

of  Z  for  various  values  of 

^  W  =Z  W  (>W  ) 

a  a  a,  o' 

(1-a)  1001?=  100 j  dP(w|H^)  =  100 J  Wj^(w|H^)  dW  , 


(14.19) 


and  sample-size  M,  from  which  one  at  once  obtains 


a  =  ;  w^(w|h^)  dW 


(U.20) 


The  upper  limit  on  Z  =  W  is  unity,  and  there  is  a  positive  nonzero  lower 
limit,  W  .  Specifically,  here,  the  test  statistic  W  is  scale-(M7Y)  and 
origin-(~X)  invariant,  and  has  a  p.d.  which  depends  only  on  sample  size  M, 
for  samples  from  a  normal  population.  Furthermore,  W  is  statistically 

p 

independent  of  S  and  of  (X)  ,  the  mean  of  the  ordered  sample  values. 

(See  p.  595,  Ref.lS.)  Note  here  that  small  values  of  =  W  are 

sigi-ificant— i.e. ,  indicate  non -normality.  However,  by  (!4.19)  and  (It. 20), 
the  test  remains. 


decide  H  :  normality  of  sample,  if  Z  ^  =  W  <  W  (^^) 

_ o _ _ _ sample  sample  a  Of 


decide  H non-normality  of  sample,  if  Z  ,  >  V/  . 

_ 1 _ _ sample  a 


(1^.21) 


Again,  we  proceed  in  similar  fashion  for  each  range  t^,  including  those 
cover! nr  the  teirget. 

V.  First-  and  Second-Order  Sample  Statistics; 


Frequently,  the  lower-order  moments  of  the  observed  data  provide 
tsef’il  information  about  the  process  and  possible  indication  of  stability. 


UO 


stationarity,  etc.  In  particular,  if  the  corresponding  infinite  population 
moments  are  known  we  can  determine  how  finite  sample-size  influences  their 
measurement . 


If  {X(t))  Is  the  observed  reverberation  (emd  tai'get)  process, 
sisting  of  M  representations,  statistics  of  interest  to  us  are: 


1.  The  sample  mean: 

M 

_  X 
JXP  "  " 


X^^^(t)  -  mg("X  of  Wilks  (Ref.  5),  Sec.  (8.2)); 


2.  The  sample  intensity: 


J=i 


3.  The  sample  covariance; 


x^'^^t)  =  X^^^t)  -  (X(t)> 


exp 


We  also  have  the  following  relations: 


J=i  d=i 


with 


M 


J=1 


4l 


con- 

(5-1) 

(5.2) 

(5.5a) 

(5.5b) 

bM) 

(:7.4b) 


t.!'.''  '.inual  r.'unplp  variance  (l\of.  Eq.  R.?.6)  •  Wn  have  also 


M(x(t)^ 


,^P:-  ^  or  (x^>  -  + 

[-1  exp  M  X 


(‘3.140) 


We  assume  in  the  above  that  in  the  set  of  representations  the  X^'''^ 

rojrosent  independent  random  variables,  with  the  same  p.d.'s.  (This 
ass  imption,  of  course,  must  be  justified,  by  procedures  like  those  discussed 
in  Sec.  ’I  above.) 


X  --  (®  -  population)  ensemble*  mean; 


2  2  * 

,  ■  X  -  X  rr  (m  _  population)  ensemble  variance; 


(5.5) 


Kj^(t^,t2)  =  xXt^TxTt^  =  (Xj^-Xj^)(Xg-X2)  =  (00  -  population)  ensemble  covariance. 


We  want  now: 

1.  The  (ensemble)*  mean  of  the  sample  mean:  this  shows  whether 
there  is  any  departure  from,  or  "bias"  between,  the  two  measures,  i .e . , 
between  the  sample  moan  vs .ensemble  mean; 

?.  The  (ensemble)*  variance  of  the  sample  mean:  this  pives  a 
measure  of  spread  in  the  experimental  mean  as  a  functio.''  of  ensemble  size 
and  the  true  variance;  and 

3.  The  (ensemble)  covariance  of  the  sample  mean:  this  provides 
a  measure  of  correlation  between  means  at  different  times  (ranpes)  and 
some  idea  of  how  close  samples  (in  time)  can  be  taken  before  such 
correlation  becomes  sipnificant. 


'"Husemble"  here  refers  to  ^  he  theoretical,  infinite  population  ersemblc, 
"ot  to  the  subonsemble  of  the  experimental  data  set. 
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From  previous  work*  we  find,  finally,  that  (with  x  =  X-X) 


1.  Varisince  of  the  experimental  variance  rxj^=x^=x^=x^;(x^-x(tj^))  ]; 

— E  —2 

var{x,2\  .4=y  .  (5.6a) 

\  1  /exp  \  1  /  exp  \  1  >fexp  M 

(Also:  (s/)^  =  5  K  -  dS) 

[cf.,  Wilks  (Ref.  5),  Eqs.  8.2.?,  8.2.9.]) 


Variance  of  the  experimental  covariance  [Xj^=Xj;  X2=x^]: 

2 


vtr 


(V^exp  ^  V-exp 


exp 


2  2  „2 
^1  ^2  '  ^x-12 

M 


with 


''x-12  ' 


(5.7a) 


(5.7b) 


5.  Covariance  of  the  experimental  variance  [xj^=x^;  Xj=Xj^]; 
K*i%xp  ■^j)exp^^(^2  )exp  "  <^2)exp^  " 


— TT  ”1 

-  ^1  ^3  -  ^1  •  ^3 
M 


(5.8) 


*D.  Middleton,  current  ONE  Research  Notes. 
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ft.  Covariarii'c  of'  the  experimental  eovariaiice; 


(t,  ,...,t,,)  -  K  (t,  .t,J  •  K  (t,,t.  )  -  K  (t,  ,t,J  .  K  (t^,t,  , 

X.  _  1  X  1'  P'exp  X  5  h'exp  x  1’  2  exp  x  3  4'exi 


x-exp 


(5.9a) 


-  \-i2  •  V3i- 

M 


(5.9b) 


By  special  ix  Ltip  (“'.'0  wo  (.'ot  tlio  other  cases  (*3.6)  throurh  directly, 

When  the  {  roco.ss  x  is  ^'aussialI  wo  may  use  Eq.  7.?9a  ol’  Hef.  1  to  obtain 
directly 


] .  var  '  x,  ) 

\  f  A'Xf' 


=  2  x^^  /M 


(s.lOa) 


2.  var  /v,  x^\ 

>  1  2/s 


ir  2  ^  2  2 

4v"AC.  X  C 


(5.10b) 


5.  (IC  ) 

X  12 


2  x^Xj  /M  , 


(5.10c) 


X  123 


(K  •  K  ol  +  K  •  K  ,,,)/M. 
x-l2  x-2''  x-l>  x-lt  ' 


(b.lOd) 


A  practical  difi'icultv  with  uslnp  these  moment  relations  is  the  fact 
that  very  often  we  do  not  know  the  ensemble  aver  ape  in  question,  e .  ,  x  , 
K  etc.  A  number  of  partial  resolutions  of  this  difficulty  is: 


1. 


2 

Use  the  sample  statistics  involved,  e.g. ,  for  use 

d 

etc.,  with  the  largest  sample  sizes  available; 


2.  Use  the  alternative  approach  of  making  repeated  observations  on 
subensembles  of  the  data,  and  estimate  the  variance  (and  mean) 
between  samples  (of  these  subensembles)  by  (5*6a),  now  given  by 


k=l 


k=l 


exp 


(5.11) 


with 


^  (1) 
d=i 


(5.12) 


(0^ 

where  is  the  statistical  form  in  question,  e.g. ,  =  x  , 

with  J  =  d ',...,  y  +  J. ) .  Of  course,  S  ^  is  itself  a  random 
veiriable,  whose  p.d.'s  depend  on  still  other  unknown  parameters. 
(For  some  simple  applications  and  a  discussion  of  the  implementing 
circuitry,  see  Korn  (Ref.  19).) 


VI.  Simulation  and  Reality; 

Simulation- -in  tank,  by  computer,  in  a  lake--provides  a  powerful 
tool  for  obtaining  quantitative  results  under  varying  degree."  of  control. 
The  crucial  problem  in  applying  the  results  of  simulation  is  to  determine 
the  extent  to  which  the  results  are  truly  representative  of  the  correspond¬ 
ing  real-life  situations.  Statistical  methods  appear  to  be  the  needed 
link  between  simulation  and  the  real-data  environment.  We  very  briefly 
sketch  the  approach: 
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I.  First,  verify  that  the  ensemble  data  from  simulation  constitute 
a  valid  enseml'le  (tcstint;’  for  independence  and  homogeneity  of  the  member 
representations,  (cf.  Sec.  j) . 

il.  Next,  establish  the  valid  ensemble  size  for  the  "real"  data  to 
be  studied. 

5.  Simulation,  of  course,  must  be  carried  out  under  as  close 
similarity  to  the  actual,  known  conditions  under  which  the  real-data 
ensemble  was  established.  In  practise,  one  may  conveniently  be  able  to 
treat  a  range  of  conditions  in  the  simulation,  and  thus  "straddle"  the 
real  environment's  constrain! s. 

1*.  Next,  "calibrate"  the  simulation  in  terms  of  the  real  data  set: 
specifically,  for  reverberation  without  target,  we  use  the  statistical 
measures  of  the  real  data  to  establish  the  form  and  relative  scale  of 
tne  distributions  and  their  (lower-order)  moments  ("goodness -of -fit" 
tests  and  estimation,  etc.).  It  is  understood  that  the  conditions  of 
simulation  axe  matched  as  closely  as  possible  to  those  under  which  the 
real  data  were  taken.  Then,  we  seek  isomorphisms  between  the  results 
from  the  real  data  and  the  hoped-for  equivalent  simulation.  The  absolute 
s-’ale  is  not  Important,  but  isomorphisms  and  departures  from  them  on  the 
part  of  the  simulated  data  then  provide  quantitative  and  qualitative  clues 
an  to  possible  differences  (and  relative  equivalon 'os)  between  the  real  and 
simulated  models. 

5.  Once  an  acceptable  calibration  of  the  simulation  has  been  achieved, 
includini'  known  (relative)  dei  artures  from  the  results  of  the  real  data,  we 
can  proceed  to  exploit  the  simulated  model  for  other  pro])erties,  checking 
back  with  the  real  data  when  feasible.  This  is,  of  course,  a  kind  of 
boot-strap  operation,  bolt  it  is  typical  of  the  kind  of  Joint  statistical. 


experimental  interaction  aided  by  physical  insiRhts  appropriate  to  the 
problem  at  hand,  that  must  be  devised  if  one  is  to  relate  simulation  and 
reality  in  a  convincing  fashion.  The  details  of  what  statistical  properties 
(usually  first-order  probability  densities  and  first-  and  second-order 
moments  to  begin  with)  are  particularly  useful,  will  depend  on  the  type 
of  measurements  undertaken.  In  any  case,  statistical  methods  provide  the 
needed  connection  between  reality  and  simulation,  so  that  the  techniques  and 
results  of  the  latter  may  be  relied  upon  with  confidence  (determined  by 
the  significance  levels,  a,  chosen). 

VII.  Concluding  Remarks 

In  the  preceding  sections  we  have  briefly  described  some  of  the 
principal  features  of  statistical  tests  in  general  and  specific  tests  in 
particular,  for  determining  whether  or  not  the  observed  data  set  represents 
a  valid  ensemble,  whether  two  data  sequences  belong  to  the  same  underlying 
process,  i.e. .  have  the  same  distributions,  and  whether  a  data  set  has  a 
particuleur  (first-order)  distribution.  Examples  of  both  parametric  (i.e. , 
distribution-dependent)  and  nonparametric  (distribution-free)  tests  are 
provided.  Generally,  the  nonparametric  tests  (Kolmogorov-Smirnov;  Runs, 
W-test,  etc.)  are  the  most  reliable,  since  they  require  few,  if  any,  un¬ 
available  a  priori  statistical  data.  The  Kolmogorov-Smirnov  tests  are 
also  among  the  more  powerful  (larger  1-P  values)  even  for  small  samples, 
with  the  x^-tests  breaking  down  when  sample-size  is  too  small. 

All  of  the  above  is  cast  in  the  particular  framework  of  measuring 
the  characteristics  of  reverberation  data,  which  can  contain  a  target  at 
certain  remges,  emd  is  intended  to  provide  a  working  preliminary  for  the 
analysis  of  data  in  such  situations,  with  attention  to  target  properties. 
Included,  also,  is  a  short  "program”  for  relating  simulated  and  "real" 
data,  which  is  a  critical  problem  in  ultimately  practical  applications. 

The  present  discussion  is  far  short  of  offering  a  comprehensive  account 


47 


of  "best"  rests  for  all.  such  environments:  one  next  step  hero  is  to 
extend,  from  working  experience,  this  small  collection  of  procedures,  to 
still  others  that  may  prove  particularly  useful  in  future  experiments. 

Finally,  we  emphasize  that  statistical  measiures  of  data  by  themselves 
can  be  worse  than  useless:  they  can  be  seriously  misleading*.  We  must 
always  embed  our  statistical  analysis  as  deeply  as  we  can  in  the  phys ical 
background  of  the  experiment,  and  use  this  background  as  fully  and  carefully 
as  possible  in  our  choice  of  data  selection  processing,  choice  of  tests, 
and  interpretation.  In  this  important  sense  practical  statistical  anal/sis 
remains  an  art,  and  is  not  purely  a  technical  device  to  be  used  unthinkingly. 
In  a  later  memorandum,  we  hope  to  extend  the  above,  stressing  the  results 
for  actual  experiments. 


*See,  for  example,  the  remarks  of  W.  Feller,  "Are  Life  Gcienllsts  Over¬ 
awed  by  Statistics?"  p.  2li  of  "Scientific  Research,"  Vol.  h.  No.  i 
February  5,  1969  (published  by  McGraw  uill) . 


kS 


APPEISIX 


TABLB  1.  TABU  OF  CRITICAL  VALUB  OF  CHI 


*  Ttbltl  ii  tbri'lc«d  (rwiB  T«M«  IV  •(  FUhtt  ud  YMm:  ftatMfcaTtaWti  /Wr  Hritfietl,  tfrittUlurtl, 
uti  mtiital  nutrtk,  puMiahtd  b|r  OlUtr  m4  Bajrd  I<ld..  RdiDburglt.  by  p«tnitlwiMi  id  llii  wibvrt  Mid 


^MLihtri. 
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AFPEISIX 

TABtE  2.  TABLE  OF  CRITICAL  VALUES  K  of  (M/2)Z  ,  IE  THE 

o  ■•■pie 

KOIitOOOROV-SMIRIOV  WO-SAMFLE  TEST 


(Saall  ■Mplet)** 


-  Mj 

Ono-Uiled  teat* 

Two-UtI«d  tcatt 

-  M/2 

«  -  .03 

0 

1 

a  ■■  .03 

q 

1 

• 

3 

3 

^.a 

1 

4 

4 

— 

4 

5 

4 

3 

5 

3 

0 

S 

C 

3 

0 

7 

5 

0 

c 

S 

3 

6 

!  0 

7 

9 

" 

(i 

7 

10 

0 

7 

7 

8 

11 

0 

8 

7 

• 

13 

0 

8 

7 

8 

13 

7 

8 

7 

•  9 

M 

7 

s 

8 

0 

IS 

7 

9 

8 

9 

1C 

7 

0 

8 

10 

17 

8 

V 

8 

10 

IS 

8 

10 

0 

10 

19 

8 

10 

0 

10 

20 

8 

10 

0 

11 

21 

8 

10 

0 

11 

22 

9 

11 

9 

11 

33 

0 

11 

10 

11 

2t 

9 

11 

10 

13 

2S 

0 

11 

10 

13 

20 

9 

11 

10 

13 

27 

9 

13 

10 

13 

28 

10 

13 

11 

13 

29  1 

10 

12 

11 

13 

30  i 

10 

13 

11 

13 

3S 

11 

13 

13 

40 

11 

14 

13 

••At  aodlflcd  froi  (Rtf.  6,  pp.  2T6) 

"From  NONPAHAMETRIC  TTATISTICS:  FOR  THE  BEHAVIORAL  SCIHICES 
by  lUdney  Siegel,  ropyrlght  0  1956  by  McGraw-Hill,  Inc. 
UsHd  with  perm'ssion  of  McGraw-HUl  Book  Coapany." 


APPOISIX 


TABLE  3.  TABLE  OF  CRITICAL  VALUES  Z  OF  Z  ,  IR  THE 

n  lupl* 

KOLMOOOROV-SMIRHOy  TWO-SAMPLE  TEST 


(Large  laaplet:  two-tall«d  teat)*  (Hj,  Mj  t  bO) 


1 

Ltval  of  gigoiAMBM 

a 

Value  of  ao  large  aa  to  eall  fee  rejeetlea 

of  at  the  laOteated  level  of  el^lftetaea, 

where  -  BaalM  |yx>,  -  a^(X),| 

.10 

Za-1.«\^ 

M 

^ _ I 


.01 

i.oV^ 

1 

.001 

■IMlfiaA  froB  Salgal  (Raf  ■  6,  pR.  2T9) 

"FroB  HORPARAMETRIC  STATISTICS:  FOR  THE  BEHAVIORAL  SCIERCES 
by  Sidney  Siegel.  Copyright  ©  1956  by  HcOrav-Hill,  Inc. 
Ueed  with  pernleelon  of  McOraw-Hill  Book  Company. " 
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APPENDIX 


TABLE  1*.  TABLE  OF  CRITICAL  VALUES  OF  R  IN  THE  RUNS  TEST* 

Given  in  the  todies  or  Table  It  and  Table  5  are  various  critical 
values  of  H  for  various  values  of  M|  and  Mj.  For  the  one-sample 
runs  teat,  any  value  of  R  which  is  equal  to  or  smaller  than  that 
shown  in  Table  U  or  equal  tc  or  larger  than  that  shown  in  Table  5 
is  significant  at  the  .05  level.  (For  the  Wald-Wolfowitz  two- 
sample  runs  test,  any  value  of  R  which  is  equal  to  or  smaller 
than  that  shown  in  Table  li  is  significant  at  the  .05  level.) 


Table  U 


2 

3 

4 

5 

0 

7 

8 

2 

10  11 

12 

13 

14 

13 

16 

17 

18 

19 

20 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

3 

2 

2 

2 

2 

2 

2 

2 

2 

2 

3 

3 

3 

3 

3 

3 

4 

2 

2 

2 

3 

3 

3 

3 

3 

3 

3 

3 

4 

4 

4 

4 

4 

S 

2 

2 

3 

3 

3 

3 

3 

4 

4 

4 

4 

4 

4 

4 

3 

3 

3 

2 

2 

3 

3 

3 

3 

4 

4 

4 

4 

5 

3 

3 

5 

3 

3 

0 

6 

7 

2 

2 

3 

3 

3 

4 

4 

5 

3 

3 

3 

5 

6 

6 

0 

6 

0 

6 

8 

2 

3 

3 

3 

4 

4 

3 

3 

3 

0 

0 

0 

0 

0 

7 

7 

7 

7 

0 

2 

3 

3 

4 

4 

5 

3 

3 

C 

0 

0 

7 

7 

7 

7 

8 

8 

8 

JO  j 

S 

3 

3 

4 

5 

3 

3 

0 

0 

7 

7 

7 

7 

8 

8 

8 

8 

0 

11  1 

I 

1 

2 

3 

4 

4 

3 

3 

0 

0 

7 

7 

7 

8 

8 

8 

9 

0 

9 

9 

12 

1  ^ 

2 

3 

4 

4 

5 

6 

0 

7 

7 

7 

8 

8 

8 

0 

9 

0 

10 

10 

13 

!  2 

2 

.3 

1 

S 

5 

0 

6 

7 

7 

8 

8 

0 

0 

9 

10 

10 

10 

10 

It 

2 

2 

3 

4 

3 

5 

6 

7 

7 

8 

8 

0 

0 

0 

10 

10 

10 

11 

11 

IS 

2 

3 

3 

A 

3 

6 

C 

7 

7 

8 

8 

0 

0 

10 

10 

11 

11 

It 

12 

16 

2 

3 

4 

4 

5 

0 

C 

7 

8 

8 

0 

0 

ID 

10 

11 

11 

11 

12 

12 

17 

2 

3 

4 

4 

S 

0 

7 

, 

8 

0 

0 

10 

10 

11 

11 

11 

12 

13 

13 

18 

2 

3 

4 

S 

5 

6 

7 

8 

8 

0 

0 

10 

10 

11 

It 

12 

13 

13 

13 

10  j 

2 

3 

4 

8 

0 

0 

m 

4 

S 

8 

9 

10 

10 

11 

11 

12 

12 

13 

13 

13 

20  1 

2 

3 

4 

«» 

6 

C 

7 

8 

0 

9 

10 

10 

11 

12 

12 

13 

13 

13 

14 

*Ad«pt«4  fro*  Scigel  (Rtf.  8,  pp.  252-253) 


"From  HOffPARAMETRIC  .CTATISTICS:  FOR  THE  BEHAVIORAL  ECIENCE!: 
by  Sidney  Siegel.  Copyright  0  1956  by  McGraw-Hill,  Inc. 
Used  with  permission  of  McGraw-Hill  Book  Company." 
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TABLE  5.  TABLE  OE  CRITICAL  VAIAiES  OF  R  IH  THE  RUHS  TEST 
TABLE  5 


17 

18 
10 
30 


17  18 
17  18 


It)  2t)  21 

10  30  31 

17  18  20  31  33 

17  18  30  31  33 


33  33 

32  33 
23  33 

33  31 


23  24 

24  25 

24  25 

25  25 


35  25 
35  2C 
30  20 
20  37 


20  20 
20  27 
27  27 
27  28 


*Adapttd  fro*  S«i(el  (Rtf.  8,  pp.  252-253) 


"From  llOIfPARAKETRIC  STATISTICS!  FOR  THE  BEHAVIORAL  SCIENCES 
by  Sidney  Siegel.  Copyright  0  195b  by  McGrew-Hlll,  Inc. 
Used  with  perm!  .'.Sinn  of  MoOrew-Hill  Book  Compeny." 
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TABLE  6.  TABLE  OF  PROBABILITIES  ASSOCUTED  WITH  VALUES  AS  EXTRIME  AS 
OBSERVED  VALUES  OF  Z  IR  THE  SORMAL  DISTRIBUTIOR* 

nie  body  of  the  table  gives  one-tailed  probabilities  under  ot  z. 
The  left-hand  susrglnal  column  gives  various  values  of  z  to  one  decimal 
place,  nie  top  row  gives  various  values  to  the  second  declrud  place. 
Thus,  for  example,  the  one-tailed  p  of  Z  2  .11  or  z  s  -.11  Is  p  ■  .U562. 


NONPARAMETRIC  STATISTICS:  FOB  THE  BEHAVIORAL  SCIENCES 
dney  Siegel.  Copyright  0  1956  by  McGraw-Hill,  Im.. 
with  permission  of  McGraw-Hill  Book  Company." 


■;6 


j 


'’rom 
)y  3i 
Ised 
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TABLE  7-  TABLE  OF  CRITICAL  VALUES  OF  Z  III  IBB  KOLMOOOROV-SMIRHOV 
OIE-SAMPLE  TEST* 


flunpt* 


Lerel  of  •l*nlflCMice  for  ■  ■Mrlwiia  |F(X)  -  8jj(X)| 


mam 

M 

.15 

.10 

.05 

.01 

1 

Z  ..BOO 

m 

.B30 

m 

.995 

9 

*  .684 

mSm 

.770 

.939 

1 

.  .565 

.642 

.708 

.838 

4 

.4B4 

.564 

.634 

.733 

S 

.446 

mm 

.810 

.565 

.660 

6 

.410 

.430 

.470 

.531 

.618 

7 

.381 

.405 

.438 

.486 

.577 

i 

.858 

.381 

.411 

.457 

.543 

9 

.33B 

.300 

.388 

.433 

.514 

10 

.332 

.343 

.368 

.410 

.490 

11 

.307 

.330 

.352 

.391 

.408 

» 

.395 

.313 

.338 

•  .875 

.450 

13 

.384 

.303 

.335 

.361 

.433 

14 

.374 

.392 

.314 

.349 

.418 

13 

.360 

.283 

.304 

.338 

.404 

13 

.358 

.374 

.295 

.328 

.392 

17 

.350 

.306 

.386 

.318 

.381 

18 

.344 

.359 

.278 

.309 

.871 

IB 

.337 

.353 

.373 

.301 

.363 

10 

.331 

.346 

.364 

.394 

.350 

25 

.31 

.33 

.24 

.37 

.32 

SO 

.10 

.30 

.33 

.24 

.30 

35 

.18 

.19 

.31 

.33 

.37 

Over  35 

1.07 

1.14 

1.22 

1.30 

1.6.3 

• 

VN 

VN 

VF 

VF 

VF 

*Adapted  from  Selgel  (Ref.  8,  pp.  251) 

"From  NONPARAMETRIC  STATISTICS:  FOR  THE  BEHAVIORAL  SCIHICES 
by  Sidney  Siegel,  Copyright  @  1956  by  McGrew-Hlll,  Inc. 
Used  with  permission  of  McGrew-Hlll  Book  Company." 
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APPENDIX 


TABLE  8.  COEFFICIENTS  FOR  THE  W  TEST  FOR  NORMALITY,* 

FOR  M  *  2(1)50. 


K 

2 

a 

4 

5 

4 

7 

8 

9 

10 

1 . 

•■7071 

0-7071 

0-CS72 

0-6646 

06431 

0-6233 

0-60.52 

0-5888 

0-3739 

2 

— 

•0000 

•1677 

•2413 

■2306 

•3031 

•3164 

•3244 

•3291 

3 

— 

— 

— 

•0000 

•0875 

•1401 

-1743 

•1976 

•2141 

4 

— 

— 

— 

— 

— 

•0000 

•0.561 

•0047 

•1224 

5 

— 

— 

— 

— 

— 

— 

— 

•0000 

•0399 

\  M 
i\ 

11 

13 

13 

14 

15 

16 

17 

~  18 

19 

20 

1 

»3tj0t 

0-5475 

0-5359 

0-.5251 

0-5150 

,0-30.50 

0-4968 

0-4SS6 

0-4S0.S 

0-47.14 

2 

•2315 

•3325 

•3335 

•3318 

•3306 

-3290 

•3273 

•32.53 

•3232 

•3211 

3 

•22*50 

•2347 

•2412 

•2460 

•2495 

•2521 

•2.540 

•2.533 

•2561 

•2565 

4 

-1429 

•1506 

•1707 

•1802 

•1878 

•1939 

•loss 

•2027 

•20.50 

■2065 

S 

•0695 

•0923 

•1009 

•1240 

•1353 

•1447 

•1524 

•1587 

•1641 

•1686 

4 

•■OflOO 

0-0303 

00539 

0-0727 

0-0880 

0-1005 

0-1109 

0-1197 

0-1271 

01334 

1 

— 

— 

•0000 

-0240 

•0433 

•0393 

•0725 

•0337 

•0032 

•1013 

t 

— 

— 

— 

— 

•0000 

•0106 

•03.59 

•0496 

•0612 

•0711 

4 

~ 

_ 

— 

— 

— 

•0000 

•0163 

■0303 

•0422 

10 

— 

— 

— 

— 

— 

— 

— 

•9000 

•0140 

X" 

21 

23 

S 

34 

35 

26 

rt 

28 

29 

30 

1 

(►ion 

•4590 

0-4542 

0-4  40.3 

0-4450 

04407 

0-4306 

0-4.123 

0-4-291 

0-42.54 

2 

•3185 

•3150 

•3126 

•3098 

-.1069 

•3043 

-3018 

•2992 

•2968 

•2944 

3 

•2.578 

•4371 

•2563 

•25.54 

•2343 

•2333 

•2.522 

'2310 

•2499 

•2467 

4 

•3119 

•2131 

•2139 

•2143 

•2143 

•2151 

•21.52 

•2131 

•21.50 

•2148 

3 

•1730 

•1764 

•1787 

•1807 

•1822 

•1836 

•1848 

•1857 

•1864 

•1870 

4 

••1390 

0-1443 

0-1430 

0-1.512 

0-15.19 

01 363 

0-1.584 

01601 

(M016 

0-16.10 

7 

•1093 

•11.50 

■1201 

•1245 

•1283 

•1316 

•1346 

•1372 

•1395 

•1415 

S 

••804 

•9878 

•0941 

•0997 

•1016 

•1089 

•1128 

..  -1162 

•1192 

•1219 

9 

•0530 

•0618 

•069C 

•0764 

•0823 

•0876 

•0923 

•0063 

•1002 

•1036 

10 

•0263 

•0368 

•0450 

.  -0539 

•0610 

•0672 

•0728 

•0778 

•0822 

•0862 

11 

0-0000 

0-0122 

04)228 

0-0321 

0-0403 

0-0476 

0-0540 

0-0.598 

00630 

0-0697 

U 

— 

— 

•0000 

•0107 

•0200 

•0284 

•0358 

•0424 

•0483 

•0.537 

u 

_■ 

— 

— 

— • 

•0000 

•0094 

•0178 

•0233 

•0320 

•0381 

14 

— 

— 

_ 

— 

— 

— 

.  -0000 

•0084 

•0159 

•0227 

U 

— 

—  ■ 

— 

— 

— 

•— 

— 

■0000 

•0076 

•Adapted  from  Shapiro  and  Wilk  (Ref.  18,  pp.  603-60I4) 
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TABLE  8.  COEFFICIENTS  FOR  THE  H  TEST  FOR  NORMALITY, 

FOR  M  »  2(1)50  (cont.) 


X 

31 

32 

33 

54 

35 

36 

37 

38 

30 

40 

1 

0-4220 

0-4188 

0415C 

04127 

0-400i> 

0-4068 

0-4010 

04015 

0-30S9 

0-3064 

2 

•2021 

•2808 

•2876 

•2854 

•2834 

•2813 

•2794 

•2774 

•2735 

•2737 

3 

•2475 

•24C3 

•2451 

•2430 

•2427 

•2415 

•2403 

•2301 

•2380 

•2368 

4 

•2145 

•2141 

•2137 

•2132 

•2127 

•2121 

•2110 

•2110 

•2104 

■2093 

5 

•1874 

•1878 

•1880 

•1882 

•1883 

•1883 

•1883 

•1881 

•1880 

'•1878 

« 

0-1C41 

O-lO.ll 

0-1060 

016G7 

0-1673 

0-1678 

0-1CS3 

0-1686 

O-ICSO 

0-1601 

7 

■1433 

•1449 

•1463 

•1475 

•1487 

•1490 

•1505 

•1513 

■15-20 

•1526 

• 

•1243 

•1265 

•1284 

-1301 

•1317 

•1331 

-1344 

•1356 

•I3CC 

•1376 

9 

•lose 

•1C03 

•1118 

•1140 

•1160 

•1179 

•1190 

•1211 

•1225 

•1237 

10 

•0800 

•0931 

•0961 

•0088 

•1013 

•1036 

•1050 

•1075 

•1002 

•1108 

It 

0-0739 

0-0777 

0-0812 

0-0844 

0-0873 

0-0900 

0-0924 

0-0047 

0-0967 

0-0086 

U 

•0586 

•0629 

•0660 

•0700 

•0739 

•0770 

-0798 

•0824 

•084S 

•0870 

13 

•0435 

•0485 

•0530 

•0572 

•0610 

•0645 

•0677 

•0706 

•0733 

•0750 

14 

■0280 

•0344 

•0305 

•0441 

•0484 

•0523 

■0559 

•0502 

•0022 

•0651 

IS 

•0144 

•0206 

‘0262 

•0314 

•0361 

•0404 

-0444 

•0481 

•0515 

•0546 

14 

0-0000 

0-0068 

0-0131 

0-0187 

0-0239 

0  0287 

0-0331 

00372 

0  0400 

00444 

17 

— 

— 

•0000 

•0062 

•0119 

•0172 

•0220 

•0264 

-0305 

•0343 

18 

— 

— 

— 

•0000 

•0057 

•0110 

•0158 

•0203 

•0244 

10 

— 

— 

— 

— 

— 

•0000 

•0053 

•0101 

'•0140 

20 

— 

— 

— 

— 

— - 

— • 

— 

— 

■0000 

■0049 

41 

42 

43 

44 

45 

46 

47 

'  48  > 

49 

50 

1 

0-3940 

0-3917 

0-3801 

0-3872 

0-3830 

0-3830 

0-3808 

0-3789 

0-3770 

0-3751 

2 

•2719 

•2701 

•2684 

•2667 

•2651 

•2633 

•2620 

•2604 

•2589 

•2574 

3 

•2357 

■2345 

•2334 

•2323 

•2313 

•2302 

•2291 

•2281 

•2271 

•2200 

4 

•2091 

•2085 

•2078 

•2072 

•2065 

•2038 

-2052 

•2045 

•2038 

•2032 

5 

•1876 

•1874 

•1871 

•1868 

•1805 

•1862 

•1859 

•1855 

•1851 

•1847 

4 

0-IG93 

0-1601 

0-IG05 

0-IG9.5 

0-1605 

0-1695 

0-1095 

0-1693 

0-1602 

0-I09I 

7 

•1531 

•1535 

•1539 

■1542 

•1545 

•1548 

•1550 

•1551 

•1553 

•1554 

• 

•1384 

•1392 

•1308 

•1405 

•1410 

•1413 

•1420 

•1423 

•1427 

•1430 

9 

•1249 

•1259 

•1260 

•1278 

•1286 

•1293 

-1300 

'1300 

•1312 

•1317 

10 

•1123 

•1130 

•1149 

•1160 

•1170 

•1180 

•1180 

•1197 

•1205 

•1212 

11 

0-1004 

0-1020 

0-1035 

01049 

0-IOG2 

0-1073 

0-I0S5 

0-I095 

0-1105 

Q-1II3 

12 

0891 

•0909 

•0027 

•0943 

•0059 

•0972 

•0980 

■0998 

•1010 

•1020 

13 

•0782 

•0804 

•0824 

•0S42 

•0860 

•0876 

•0892 

•0906 

•0019 

•0032 

14 

■0677 

■0701 

•0724 

•  -0745 

•0763 

•0783 

•0801 

•0817 

•0832 

•0846 

IS 

•0575 

•0602 

•0628 

•0651 

■0673 

•0004 

•0713 

•0731 

■0748 

•0704 

14 

0-0476 

0-0506 

0-0534 

0-0560 

0-0o84 

0-0607 

0-0628 

0-0648 

0-0667 

0-0685 

17 

•0379 

•0411 

0442 

•0471 

•0497 

•0522 

■0346 

•0568 

•0583 

•OOOH 

18 

•0283 

•0318 

•0352 

•0383 

•0412 

•0439 

•0465 

•0480 

•0511 

•0532 

10 

•0188 

•0227 

■0263 

•0296 

•0328 

■0357 

•0383 

•0411 

•0436 

•0459 

20 

•0094 

•0130 

•0175 

•0211 

•0245 

•0277 

•0307 

•0333 

•0361 

•0386 

21 

0-0000 

0  0045 

0-0087 

0-0136 

0-0103 

0-0197 

0-0229 

0-0239 

0-0288 

0-0314 

22 

— 

•0000 

•0042 

•0081 

•0118 

■0153 

•0185 

•0215 

•0244 

23 

— 

— 

— 

— 

•0000 

•0039 

•0078 

•0111 

•0143 

•0174 

34 

— 

— 

— 

— 

— 

— 

•0000 

•0037 

•0071 

•0104 

25 

— 

— 

— 

— 

•  — 

— 

— 

— 

•9000 

•0035 

^9 


TABLE  9.  PERCEBTAGE  POINTS  OF  THE  W  TEST*  FOR  M  -  3(1)50 

Low! 


M 

3  ■  0.99 

0.98 

0.95 

0.90 

0.50 

o.Uo 

0.05 

0.02 

0.01 

X  /o.oi 

0.02 

0.05 

0.10 

0.50 

0.90 

0.95 

0.98 

0.99 

W  »  Z  0.753 

0.7‘:6 

0.767 

0.789 

0.759 

0.998 

0.999 

1.000 

1.000 

4 

•OH* 

•7«7 

•748 

•702 

•035 

•087 

•002 

•006 

•007 

S 

•OSH 

•715 

•762 

•806 

•927 

•079 

•086 

•091 

•093 

4 

0-71.1 

0  74.1 

0-I8S 

0-826 

0  927 

0074 

0081 

0-086 

0-080 

7 

•7311 

•760 

•803 

•838 

•928 

•072 

•079 

•085 

•088 

• 

•749 

•778 

•818 

•8.71 

•032 

•972  - 

•978 

•984 

•J87 

9 

•704 

•701 

•829 

•8.79 

•935 

•072 

•978 

•084 

-986 

10 

•781 

•806 

•842 

•860 

•038 

•972 

•078 

•983 

•986 

11 

0-7UJ 

0817 

0-830 

0  876 

0-0(0 

0073. 

0-079 

0084 

0-086 

13 

•80.-I 

•828 

-830 

•833 

•943 

•973 

•979 

•084 

•986 

13 

•814 

•837 

•866 

•889 

-943 

•974 

•079 

•984 

•986 

14 

•82-1 

•846 

•874 

•893 

•947 

•973 

•080 

•084 

•986 

15 

•833 

•853 

•881 

•901 

•050 

•975 

•980 

•984 

•987 

16 

0-844 

0-863 

0-8S7 

0-906 

0-032 

0-076 

0-081 

0-085 

0987 

17 

•831 

•860 

•892 

•910 

•934 

•977 

-081 

•085 

•087 

18 

•838 

•874 

•897 

•914 

•038 

•078 

•982 

•986 

•088 

19 

•0C3 

•879 

•90t 

•017 

•937 

•978 

•982 

•086 

•088 

30 

•808 

•884 

•005 

•020 

•050 

•079 

•083 

•986 

•988 

31 

0-873 

0-888 

0-008 

0-023 

0-060 

0-980 

0-983 

0-087 

0-089 

33 

•878 

•802 

•Oil 

•926 

-061 

•980 

-084 

-987 

•989 

33 

•881 

•895 

•914 

•928 

•962 

-081 

•084 

•987 

•080 

34 

•884 

•898 

•916 

•030 

•063 

•981 

•984 

•987 

•989 

35 

•888 

•901 

•918 

•931 

•964 

•981 

•085 

•988 

•980 

36 

0-891 

0-904 

0-920 

0-033 

0-003 

0-082 

0-985 

0-688 

0-989 

r 

•804 

•900 

•923 

•935 

•005 

-982 

•983 

•088 

•900 

38 

•896 

•90S 

•024 

•936 

•906 

-982 

•985 

•988 

•990 

39 

•898 

•910 

•926 

•937 

•960 

•982 

•985 

-088 

•900 

30 

•900 

•012 

•927 

•939 

-007 

•083 

•983 

•088 

•900 

31 

0-002 

0-014 

0-929 

0-940 

0-907 

0083 

0-986 

.  0-088 

0-900 

33 

•904 

•913 

•030 

•941 

•068 

•983 

-086 

•988 

•990 

33 

•900 

•917 

•931 

•942 

•968 

•983 

•986 

•989 

•990 

34 

•003 

•019 

•933 

•043 

•069 

•983 

-986 

•089 

-9DO 

35 

•910 

•920 

•934 

•044 

•960 

•984 

•986 

•980 

•900 

36 

0012 

002-2 

0-035 

0-943 

0-970 

0-984 

0-936 

0-089 

0-99'> 

37 

•014 

•024 

•930 

•946 

•970 

•084 

•087 

•989 

•090 

38 

•910 

•023 

•938 

•947 

•971 

•984 

•987 

•089 

•990 

39 

•917 

•027 

•939 

•048 

•971 

•084 

•987 

•980 

•991 

40 

•919 

•028 

•940 

•049 

-972 

•983 

•987 

•980 

•901 

4! 

0-920 

0029 

0-941 

0-050 

0072 

0-085 

0-037 

0-089 

0-901 

43 

•922 

•930 

•942 

•051 

•972 

•985 

-037 

•089 

•901 

43 

•923 

•032 

•943 

•951 

•973 

•085 

•987 

•900 

•991 

44 

•924 

•033 

•944 

•952 

•973 

•985 

•987 

•900 

•991 

45 

•926 

•934 

•945 

•953 

•973 

•985 

•088 

•990 

•991 

46 

0-927 

0-935 

'  0-943 

0-033 

0-074 

0-085 

0-988 

0-990 

0-091 

47 

•928 

•936 

•940 

•934 

•974 

•985 

•988 

•900 

•901 

48 

•020 

•037 

•947 

•054 

•974 

•985 

•988 

•900 

•901 

49 

•020 

•937 

•947 

•953 

•974 

•985 

•988 

•900 

•901 

80 

•030 

.•038 

•947 

•955 

•974 

•985 

•088 

.  -900 

•991 

•Adapted  from  Shapiro  and  Wilk  (Ref.  l8,  pp.  605) 
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